close

Вход

Забыли?

вход по аккаунту

JP2012186551

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2012186551
To improve the quality of sound collection. A control device for controlling an image acquiring
unit capable of panning left and right and a sound collecting unit capable of changing a sound
collecting range, which is acquired by the position of the image acquiring unit and the image
acquiring unit Between the position of the subject in the image to be captured, the direction of
the pan of the image acquisition means, the distance between the position of the image
acquisition means and the position of the sound collection means, and the image acquisition
means A sound collection range changing means for changing the sound collection range of the
sound collection means using the direction of the position of the sound collection means with
reference to the position of. [Selected figure] Figure 2
Control device, control system and control method
[0001]
The present invention relates to a technology for controlling a sound collection device.
[0002]
Conventionally, there is a technology of a video conference apparatus in which the directivity
characteristic of a microphone and the shake angle and zoom angle of a camera are interlocked
by a microphone built-in type video camera or the like (Patent Document 1).
[0003]
Japanese Patent Application Laid-Open No. 10-155107
04-05-2019
1
[0004]
However, in the above technology, since the camera and the microphone are integrated, the
subject (participant) who can take a picture when the microphone is placed at the center of the
conference table etc. in order to improve the sound collection quality in the video conference etc.
It is limited.
[0005]
An object of the present invention is to provide a technique for improving the quality of sound
collection.
[0006]
In order to solve the above problems, a control device according to the present invention is a
control device that controls an image acquiring unit capable of panning to the left and right and
a sound collecting unit capable of changing a sound collecting range, The distance between the
position of the acquisition means and the position of the subject captured in the image acquired
by the image acquisition means, the direction of the pan of the image acquisition means, the
position of the image acquisition means and the position of the sound collection means And
sound collection range changing means for changing the sound collection range of the sound
collection means using the distance between the two and the direction of the position of the
sound collection means with reference to the position of the image acquisition means. It is
characterized by
[0007]
Further, a control system according to the present invention includes an imaging device
including pan operation means to the left and right, a sound collection device including change
means of a sound collection range, a control device for controlling the imaging device and the
sound collection device. The control device is configured to control the distance between the
position of the imaging device and the position of the subject captured in the image acquired by
the imaging device, the direction of the pan of the imaging device, and the position of the
imaging device. A sound collection instruction to change the sound collection range of the sound
collection device using the distance between the position of the sound collection device and the
direction of the position of the sound collection device based on the position of the imaging
device A sound range change instruction unit is provided.
[0008]
Further, the imaging apparatus according to the present invention comprises: pan operation
04-05-2019
2
means for performing a pan operation to the left and right; angle of view change operation
means for performing an operation for changing an angle of view being photographed; Pan
direction information acquiring means to be acquired, distance information acquiring means for
acquiring distance information for specifying a distance to an object, angle of view information
acquiring means for acquiring angle of view information for specifying an angle of view being
photographed, and An output unit that outputs pan direction information, the distance
information, and the angle-of-view information; and the output unit ends any operation of the
pan operation unit and the angle-of-view change operation unit. When the predetermined time
has elapsed, the output is performed.
[0009]
A control method according to the present invention is a control method of a control device that
controls an image acquisition unit capable of panning to the left and right and a sound collection
unit capable of changing a sound collection range, and the control device is a control method. A
distance between a position of the image acquisition means and a position of an object captured
in the image acquired by the image acquisition means, a direction of pan of the image acquisition
means, a position of the image acquisition means, and the sound collection A sound collection
range changing step of changing a sound collection range of the sound collection means using a
distance between the position of the means and a direction of the position of the sound collection
means with reference to the position of the image acquisition means It is characterized by
carrying out.
[0010]
According to the present invention, it is possible to improve the quality of sound collection while
enabling the camera and the microphone to be disposed independently.
[0011]
It is a figure showing the example of composition of the teleconference system concerning the
embodiment of the present invention.
It is a figure which shows the function structural example of a video conference system.
It is a figure which shows the hardware constitutions of a video conference system.
04-05-2019
3
It is a figure showing composition of a microphone array device.
It is a figure which shows the example of arrangement ¦ positioning of a video conference
system.
It is a figure which shows the structural example of a microphone array position table.
It is a figure which shows the structural example of a microphone array output level table.
It is a sequence diagram of a microphone array setting process.
It is an explanatory view explaining an example of a screen of initialization processing of
microphone array position setting.
It is an explanatory view explaining a mechanism of microphone array setting calculation.
It is a figure which shows the structural example of an output level adjustment table. It is an
explanatory view explaining a mechanism of microphone array output level adjustment.
[0012]
Hereinafter, a video conference system 1 including a microphone control device to which the
first embodiment according to the present invention is applied will be described with reference to
the drawings.
[0013]
FIG. 1 is a diagram showing a video conference system 1 according to an embodiment of the
present invention.
04-05-2019
4
The video conference system 1 in the present invention includes a server 80, and a conference
terminal A20, a conference terminal B21, and a conference terminal C22 which can be connected
to the server 80 via the network 70. Basically, it is assumed that the conference terminal A20 is
disposed in the conference room A10, the conference terminal B21 is disposed in the conference
room B11, and the conference terminal C22 is disposed in the conference room C12.
[0014]
A camera A30 capable of communicating with the conference terminal A20, a microphone array
A40, a display A50, and a speaker A60 are connected to the conference terminal A20. The
conference terminal A20 can operate as a microphone control device.
[0015]
The camera A30 is a so-called image acquisition device, such as a video camera, which acquires
video information as digital data and transmits it to the conference terminal A20, receives an
instruction of a shooting range from the conference terminal A20, and performs shooting.
[0016]
The microphone array A40 collects the surrounding sound, acquires it as digital data, transmits it
to the conference terminal A20, receives an instruction such as the sound collecting direction
and the sound collection sensitivity from the conference terminal A20, and collects sound It is.
The microphone array A40, which will be described later, is an apparatus in which a plurality of
microphones collecting a predetermined range are arranged so as to have a sound collection
range extending radially all around from the installation position of the microphone array A40. Is
possible.
[0017]
The display A50 displays and outputs display information transmitted from the connected
conference terminal A20. The display information includes video information acquired by the
conference terminal A 20 via the camera A 30 or video information etc. received from the other
04-05-2019
5
conference terminal B 21 or the conference terminal C 22 via the network 70. In addition, the
display A50 can also display a setting screen such as a setting value transmitted from the
conference terminal A20.
[0018]
The speaker A60 outputs audio information transmitted from the connected conference terminal
A20. The voice information includes voice information acquired by the conference terminal A20
via the microphone array A40 or voice information received from the other conference terminal
B21 or the conference terminal C22 via the network 70. In addition, the speaker A60 can also
output audio such as guidance transmitted from the conference terminal A20.
[0019]
A camera B31 capable of communicating with the conference terminal B21, a microphone array
B41, a display B51, and a speaker B61 are connected to the conference terminal B21. The
conference terminal B21 can operate as a microphone control device.
[0020]
The camera B31 is a so-called image acquisition device, such as a video camera or the like, which
acquires video information as digital data and transmits it to the conference terminal B21,
receives an instruction of a shooting range from the conference terminal B21, and shoots.
[0021]
The microphone array B41 collects the surrounding sound, acquires it as digital data, transmits it
to the conference terminal B21, receives an instruction such as the sound collecting direction and
the sound collection sensitivity from the conference terminal B21, and collects sound It is.
The microphone array B41, which will be described later, is an apparatus in which a plurality of
microphones collecting a predetermined range are arranged so as to have a sound collection
range extending radially all around from the installation position of the microphone array B41. Is
possible.
04-05-2019
6
[0022]
The display B51 displays and outputs display information transmitted from the connected
conference terminal B21. The display information includes video information acquired by the
conference terminal B21 via the camera B31 or video information received from the other
conference terminal A20 or the conference terminal C22 via the network 70. In addition, the
display B51 can also display a setting screen such as a setting value transmitted from the
conference terminal B21.
[0023]
The speaker B61 outputs audio information transmitted from the connected conference terminal
B21. The voice information includes voice information acquired by the conference terminal B21
via the microphone array B41 or voice information etc. received from the other conference
terminal A20 or the conference terminal C22 via the network 70. In addition, the speaker B61
can also output voice such as guidance transmitted from the conference terminal B21.
[0024]
A camera C32 capable of communicating with the conference terminal C22, a microphone array
C42, a display C52, and a speaker C62 are connected to the conference terminal C22. The
conference terminal C22 can operate as a microphone control device.
[0025]
The camera C32 is a so-called image acquisition device, such as a video camera, which acquires
video information as digital data and transmits it to the conference terminal C22, receives an
instruction of a shooting range from the conference terminal C22, and shoots.
[0026]
The microphone array C42 collects the surrounding sound, acquires it as digital data, transmits it
04-05-2019
7
to the conference terminal C22, receives an instruction such as the sound collection direction
and sound collection sensitivity from the conference terminal C22, and collects sound It is.
The microphone array C42, which will be described later, is an apparatus in which a plurality of
microphones collecting a predetermined range are arranged so as to have a sound collection
range extending radially all around from the installation position of the microphone array C42. Is
possible.
[0027]
The display C52 displays and outputs display information transmitted from the connected
conference terminal C22. The display information includes video information acquired by the
conference terminal C22 via the camera C32 or video information etc. received from the other
conference terminal A20 or the conference terminal B21 via the network 70. In addition, the
display C52 can also display a setting screen such as a setting value transmitted from the
conference terminal C22.
[0028]
The speaker C62 outputs audio information transmitted from the connected conference terminal
C22. The voice information includes voice information acquired by the conference terminal C22
via the microphone array C42 or voice information etc. received from the other conference
terminal A20 or the conference terminal B21 via the network 70. In addition, the speaker C62
can also output voice such as guidance transmitted from the conference terminal C22.
[0029]
In the present embodiment, the network 70 is assumed to be a wide area network conforming to
TCP / IP etc. such as the so-called Internet etc. However, the network 70 is not limited to this,
and may be a LAN (Local Area Network), WAN (Wide Area Network) etc. It may be a network.
[0030]
The server 80 implements a video conference service between the conference terminal A 20, the
04-05-2019
8
conference terminal B 21 and the conference terminal C 22 via the network 70.
Specifically, for example, the server 80 configures and manages a virtual conference room
between conference terminals to be joined in a conference, and mediates communication
between conference terminals to be joined. In the present embodiment, the server 80 does not
acquire video data and audio data of a conference terminal and distributes the data to another
conference terminal, but establishes a communication path of video data and audio data between
the conference terminals. Stay in However, the present invention is not limited to this, and video
data and audio data between conference terminals may be intensively acquired and distributed to
other conference terminals participating in the conference.
[0031]
FIG. 2 is a diagram showing an example of a functional configuration of the camera A30, the
conference terminal A20, and the microphone array A40 disposed in the conference room A10 of
the video conference system 1. The conference room B11, the camera B31 for the conference
room C12, the conference terminal B21, the microphone array B41, the camera C32, the
conference terminal C22, and the microphone array C42 basically have the same functional
configuration. The display A 50, the speaker A 60, the display B 51, the speaker B 61, the display
C 52, and the speaker C 62 are not illustrated, but have a normal functional configuration.
[0032]
As shown in FIG. 2, the camera A30 includes a pan / zoom control unit 310, a camera
information notification unit 320, and a camera operation enable / disable control unit 330. The
pan / zoom control unit 310 controls the angle of view taken by the camera A 30 and the
direction in which the camera A 30 shoots, that is, the range included in the video. Specifically,
upon receiving an instruction to change the shake angle to the left or right, the pan / zoom
control unit 310 changes the shake angle of the optical device 35 described later according to
the instruction. When an instruction to change the angle of view is received, the angle of view of
the optical device 35 described later is changed according to the instruction.
[0033]
When the pan / zoom control unit 310 can not follow the pan or zoom change instruction, for
example, according to the shake angle change instruction, it exceeds the physical movable range
04-05-2019
9
of the optical device 35, or, for example, According to the instruction to change the angle of view,
if the range of the optical or image processing zoom (optical zoom, digital zoom, etc.) of the
optical device 35 is exceeded, control is performed in accordance with the changeable range.
[0034]
The camera information notification unit 320 acquires information on the shake angle (pan
direction), the distance to the subject, and the angle of view at a predetermined timing, and
outputs the information to the connected conference terminal A20.
In the present embodiment, the camera information notification unit 320 performs a shake angle
(pan direction) when the pan / zoom operation is not performed for a predetermined time (for
example, 3 seconds) after the pan / zoom operation is performed. It is assumed that information
on the distance to the end and the angle of view is acquired and output to the connected
conference terminal A20. In the present embodiment, the shake angle (pan direction) is obtained
from the difference between the direction of the base (front) and the direction of the optical axis
of the camera, and the distance to the subject is the subject on the subject at a predetermined
distance measurement point. The angle of view is an angle of view determined according to the
amount of extension of the lens unit.
[0035]
The camera operation valid / invalid control unit 330 controls valid / invalid of acceptance of an
operation from the conference terminal A20 to the camera A30. Specifically, when the camera
operation valid / invalid control unit 330 receives a camera operation lock instruction from the
conference terminal A 20, the camera operation valid / invalid control unit 330 performs
operation lock so as not to receive a pan / zoom change instruction to the pan / zoom control
unit 310. Also, upon receipt of the camera operation lock release instruction from the conference
terminal A 20, the camera operation valid / invalid control unit 330 releases the operation lock
so as to receive the pan / zoom change instruction to the pan / zoom control unit 310.
[0036]
Although not shown, the camera A30 receives an instruction to start image acquisition and
04-05-2019
10
acquires an image as in a normal camera, and an image to end image acquisition by receiving an
instruction to end image acquisition. Acquisition end control unit, image output control unit
outputting the acquired image to a storage device such as an external device such as a
conference terminal A20 or a memory card attached to the camera A30, a distance measuring
unit specifying a distance to an object, an optical device A control unit such as a focus control
unit that adjusts a focal position (focus) of 35 is provided. Also, the camera A30 can perform pan
and zoom operations by accepting pan and zoom operations from the operator (or the
conference participant) of the conference room A10 or the operator of the conference partner (or
the conference participant).
[0037]
The conference terminal A20 includes a camera state detection unit 210, a microphone array
control unit 220, a microphone array position data memory 230, a camera operation valid /
invalid designation unit 240, and a camera operation designation unit 250. The camera state
detection unit 210 detects the state of the connected camera A30. Specifically, the camera state
detection unit 210 acquires information on the shake angle (pan direction), the distance to the
subject, and the angle of view from the connected camera A30. Furthermore, when the
microphone array A40 in the image captured by the camera A30 is at a predetermined screen
position, the camera state detection unit 210 detects the microphone array A40 according to the
deflection angle of the camera A30 and the distance to the subject. Identify the location of
[0038]
The microphone array control unit 220 performs gain adjustment for each of the microphones of
the microphone array A40 according to the position of the microphone array A40, the position of
the subject of the camera A30, and the angle of view. Specifically, the microphone array control
unit 220 stores microphone array position data storing the relative position of the microphone
array A40 (the pan angle from the front direction of the camera A30 and the linear distance from
the camera A30 to the microphone array A40) The microphone position is acquired from the
memory 230.
[0039]
Then, the microphone array control unit 220 uses the direction (specified based on the shake
04-05-2019
11
angle) of the subject viewed by the camera A30 (specified based on the shake angle), the distance
to the subject, the angle of view, and the microphone position to use the microphone position
Identify the direction of the subject seen from the viewpoint. The microphone array control unit
220 outputs a microphone gain adjustment instruction including a predetermined microphone
gain adjustment value to the microphone array A40 for each microphone of the microphone
array A40 according to the direction of the subject viewed from the microphone position. In the
present embodiment, when the camera angle of view is 60 ° or more, the output level of the
microphone amplifier is not adjusted. However, when the camera angle of view is less than 60 °,
the output level of the microphone is increased or decreased to control the sound collection
range.
[0040]
The microphone array position data memory 230 stores information for specifying the position
of the microphone array A40. Specifically, the microphone array position data memory 230
includes a microphone array position table 600 shown in FIG. 6 and a microphone array output
level table 650 shown in FIG.
[0041]
The microphone array position table 600 stores the position of the microphone array. The
microphone array position table 600 stores the information specifying the distance between the
camera A and the microphone array A. The distance 601 between the camera A and the
microphone array A and the pan direction of the microphone array viewed from the front of the
camera A reference pan direction angle 602 is included. For example, when the reference pan
direction angle 602 is −5 ° , it indicates that the microphone array is present in the minus
5 ° direction from the front of the camera A30, that is, in the left direction.
[0042]
The microphone array output level table 650 stores information for adjusting the output level of
each of the microphones constituting the microphone array. The microphone array output level
table 650 includes a camera angle of view 651 which is an angle of view taken by the camera
A30, and an output level of a microphone selected as a sound collection direction set as an
output level of a microphone selected as a sound collection direction. Set the output level of the
04-05-2019
12
microphone adjacent to the selected microphone 652 and the output level of the microphone
adjacent to the selected microphone, and the output level of the other microphones excluding the
selected microphone and the adjacent microphone And the output levels 654 of the other
microphones. The output level stored in the microphone array output level table 650
corresponds to the camera angle of view, with the output level of the microphone array A 40 not
performing control of the sound collection range (360 sound collection state) as 0 dB. The output
level of each microphone amplifier is specified.
[0043]
The camera operation valid / invalid designation unit 240 instructs the camera A30 to operate
from the conference terminal A20. Specifically, camera operation valid / invalid instruction unit
240 instructs camera operation lock instruction / camera operation lock release for camera A30.
[0044]
The camera operation instruction unit 250 instructs an operation from the conference terminal
A20 on the camera A30 or another camera B31 or camera C32 in a conference via the network
70. Specifically, when the camera operation instruction unit 250 receives a pan / zoom operation
of a designated camera among cameras connected to a conference terminal participating in the
same conference, the camera operation instruction unit 250 transmits the operation target
camera. Output pan / zoom change instruction. At that time, if a camera connected to another
conference terminal is an operation target, information including a change instruction
information is sent out on the network 70 with the conference terminal as a destination.
[0045]
In addition, when the camera operation instruction unit 250 receives a pan / zoom change
instruction of the camera via the network 70, the camera operation instruction unit 250 outputs
a pan / zoom change instruction according to the change instruction to the connected camera
A30. Do. By doing this, it is possible to pan / zoom the camera of the other party's conference
room or the own conference room via each conference terminal participating in the conference.
[0046]
04-05-2019
13
The microphone array A40 is obtained from the microphone gain control unit 410, the
microphones 420a to 420l, and the microphone amplifiers 430a to 430l that adjust the gains of
the microphones corresponding to the microphones 420a to 420l, and the microphone
amplifiers 430a to 430l. And a speech coding unit 450 for converting the synthesized output
obtained by the mixer 440 into speech information.
[0047]
The microphone gain control unit 410 adjusts the gain of the microphone amplifiers 430a to
430l for each of the microphones 420a to 420l.
In the present embodiment, the microphone gain control unit 410 receives the gain adjustment
instruction of the microphones 420a to 420l from the conference terminal A 20, and adjusts the
gain of the corresponding microphone amplifiers 430a to 430l.
[0048]
The speech encoding unit 450 samples speech information output from the mixer 440 at a
predetermined frame rate, and outputs speech data. The process is a normal encoding process.
[0049]
FIG. 3 is a diagram showing an example of the hardware configuration of the camera A30, the
conference terminal A20, and the microphone array A40. The same hardware configuration is
assumed for the camera B31, the conference terminal B21, the microphone array B41, the
camera C32, the conference terminal C22, and the microphone array C42.
[0050]
In the present embodiment, the camera A30 is a so-called video camera device that operates
independently. However, the computer is not limited to this, and may be a computer classified
into a PC (personal computer), a workstation, various mobile phone terminals, a PDA (Personal
04-05-2019
14
Digital Assistant), a smartphone, and the like.
[0051]
The camera A30 includes a bus 131, an arithmetic unit 132, a main storage unit 133, a
communication unit 134, and an optical unit 135. The arithmetic unit 132 is, for example, an
arithmetic unit such as a central processing unit (CPU).
[0052]
The main storage device 133 is, for example, a memory device such as a random access memory
(RAM).
[0053]
The communication device 134 is a communication device that communicates with a device
connected to a network 70 such as the Internet or a LAN and a conference terminal A 20
connected by a dedicated cable or the like.
[0054]
The optical device 135 is a so-called imaging device including, for example, a video camera lens
group and its focus drive unit, an imager with a CCD or CMOS, etc., and an image processing unit
that performs image generation processing.
[0055]
The arithmetic unit 132, the main storage unit 133, the communication unit 134, and the optical
unit 135 are mutually connected by a bus 131.
[0056]
The above-described pan / zoom control unit 310 of the camera A30 is realized by the optical
device 135.
Further, the camera information notification unit 320 and the camera operation valid / invalid
04-05-2019
15
control unit 330 are realized by a program that causes the arithmetic device 132 to perform
processing.
The program is stored in the main storage device 133 or a ROM (not shown), loaded on the main
storage device 133 for execution, and executed by the arithmetic device 132.
[0057]
In the present embodiment, the conference terminal A20 is a so-called terminal device that
operates independently.
However, the computer is not limited to this, and may be a computer classified into a PC
(personal computer), a workstation, a server device, various mobile phone terminals, a PDA
(Personal Digital Assistant), a smartphone, and the like.
[0058]
The conference terminal A 20 includes a bus 121, an arithmetic unit 122, a main storage unit
123, and a communication unit 124.
The arithmetic device 122 is, for example, an arithmetic device such as a CPU (Central
Processing Unit).
[0059]
The main storage device 123 is, for example, a memory device such as a random access memory
(RAM).
[0060]
The communication device 124 is a device connected to the network 70 such as the Internet or
LAN, another conference terminal connected by a dedicated cable or the like, and another device
connected such as the camera A30, the microphone array A40, the display A50, the speaker A60,
etc. It is a communication device that performs communication.
04-05-2019
16
[0061]
The arithmetic unit 122, the main storage unit 123, and the communication unit 124 are
connected to one another by a bus 121.
[0062]
The camera state detection unit 210, the microphone array control unit 220, and the camera
operation valid / invalid designation unit 240 of the conference terminal A20 described above
are realized by a program that causes the arithmetic device 122 to perform processing.
The program is stored in the main storage device 123 or a ROM (not shown), loaded on the main
storage device 123 for execution, and executed by the arithmetic device 122.
[0063]
The microphone array position data memory 230 of the conference terminal A 20 is realized by
being stored in the main storage device 123.
[0064]
The microphone array A 40 is a microphone device that operates independently in the present
embodiment.
The microphone array A 40 includes a bus 141, an arithmetic unit 142, a main storage unit 143,
a communication unit 144, and a sound collection unit 145.
The arithmetic device 142 is, for example, an arithmetic device such as a CPU (Central
Processing Unit).
[0065]
The main storage device 143 is, for example, a memory device such as a random access memory
04-05-2019
17
(RAM).
[0066]
The communication device 144 is a communication device that communicates with a device
connected to the network 70 such as the Internet or a LAN and the conference terminal A 20
connected via a dedicated cable or the like.
[0067]
The sound collector 145 has a structure as shown in FIG. 4, for example.
FIG. 4 is a diagram showing the configuration of the microphone array device.
The microphone array A 40 is configured of a plurality of microphones 420 a to 420 l arranged
evenly in a ring shape.
In one microphone (the microphone 420a in FIG. 4), a reference position mark 460 as a
reference in the sound collection direction of the microphone array A40 is provided so as to be
clearly recognizable in appearance.
[0068]
Each of the microphones 420a to 420l has directivity in the sound collection range. In this
embodiment, twelve microphones 420a to 420l capable of collecting sound in the radial range of
30 degrees each on the left and right sides are combined to constitute a microphone array A40.
Therefore, it can be said that the microphone array A40 is provided with a sound collection
range at 360 degrees (all directions).
[0069]
The microphone array A40 has a sound collection range in all directions, but as described above,
the plurality of microphones 420a to 420l and the microphone amplifiers 430a to 430l that can
be adjusted for each microphone Because it is configured to include, by adjusting the gain for
04-05-2019
18
each microphone, it is possible to control the sound collection range.
[0070]
For example, in FIG. 4, the direction of the speaker (angle 461) is specified based on the
reference position mark 460, and the gain of the microphone 420e having a sound collection
range in that direction is made higher than the gains of other microphones. , It is easier to collect
the speech of the speaker, and it is possible to suppress the ambient noise.
[0071]
The arithmetic unit 142, the main storage unit 143, the communication unit 144, and the sound
collection unit 145 are mutually connected by a bus 141.
[0072]
The microphones 420 a to 420 l, the microphone amplifiers 430 a to 430 l, and the mixer 440 of
the microphone array A 40 described above are realized by the sound collection device 145.
Also, the microphone gain control unit 410 and the speech encoding unit 450 of the microphone
array A 40 are realized by a program that causes the arithmetic device 142 to perform
processing.
The program is stored in the main storage 143 or a ROM (not shown), loaded on the main
storage 143 for execution, and executed by the arithmetic unit 142.
[0073]
The above is the hardware configuration example of the video conference system 1 in the
present embodiment.
However, the present invention is not limited to this, and may be configured using other
hardware. For example, the network 70 may be a wireless network or a mobile telephone
network, and the conference terminal and the camera, microphone array, display, and speaker
04-05-2019
19
may be connectable by wireless communication such as Bluetooth (registered trademark).
[0074]
[Description of Operation] Next, the operation of the video conference system 1 in the present
embodiment will be described. FIG. 5 is a view showing an arrangement example of the camera
A30 and the microphone array A40 of the video conference system 1 in the present embodiment.
In FIG. 5, the microphone array A40 is placed on the conference desk (preferably near the center
of the conference participant), and the camera A30 is placed on one end of the conference
console. At this time, it is assumed that the microphone array A40 is installed with the reference
position mark 460 facing the front of the camera A30.
[0075]
Here, assuming that the front direction of the camera A30 is a reference (0 degrees), the right
direction is a plus direction, and the left direction is a minus direction, the installation direction
501 of the microphone array A40 viewed from the camera A30 is minus −) It is assumed that
the microphone array A40 is disposed at 5 °, that is, slightly leftward from the front of the
camera A30. The reference of the position of the microphone array A 40 is the position of the
reference position mark 460. Further, the distance 500 from the camera A30 to the microphone
array A40 is 2 m.
[0076]
The information is specified in an initialization process to be described later, and stored in the
microphone array position table 600 of the microphone array position data memory 230.
[0077]
Here, the initialization process will be described.
First, the microphone array A40 is zoomed up by the camera A30 as shown in FIG. 9 and
photographed as an initial setting image 340, and the focus is adjusted to the reference position
mark 460 of the microphone array A40. The operation is performed by the conference
04-05-2019
20
participant who uses the conference room A10 operating the optical device 35 as appropriate.
[0078]
At that time, the control unit (not shown) of the conference terminal A20 causes the display A50
of the conference room A10 to display the initial setting image 340. The control unit
superimposes and displays a scope 350 indicating a reference of the aim at the center of the
image in the initialization process. The conference participant adjusts the direction (pan
direction) of the camera A 30 so as to focus so that the scope 350 overlaps the reference position
mark 460.
[0079]
The camera information notification unit 320 identifies the distance 500 between the camera
A30 and the microphone array A40 and the installation direction 501 based on the focal length
when the subject is in focus, and notifies the conference terminal A20.
[0080]
The above is the flow of the initialization process.
By the initialization processing, the position of the microphone array A40 viewed from the
camera A30 can be specified.
[0081]
Next, microphone array control processing during a conference related to the conference room
A10 will be described using FIG. In the microphone array control processing, sound around the
subject projected by the camera is collected with high quality.
[0082]
04-05-2019
21
First, the camera A30 performs pan / zoom operation (step S001). Specifically, the camera
operation valid / invalid control unit 330 of the camera A30 is an operation of the conference
terminal A20 by a conference participant in the conference room A10, or an operation of another
conference terminal B21 or a conference terminal C22 participating in the conference. The pan /
zoom change instruction to the camera A30 according to is received. Then, when the camera
operation is valid, the pan / zoom control unit 310 controls the optical device 35 to realize the
pan / zoom operation according to the received change instruction. If the camera operation is
invalid, the camera operation valid / invalid control unit 330 discards the received change
instruction and does not respond.
[0083]
Next, the camera information notification unit 320 determines whether a camera operation has
not been performed for a prescribed time (step S002). Specifically, the camera information
notification unit 320 determines whether or not a predetermined time (for example, 3 seconds)
has elapsed since the end of the pan / zoom operation in step S001 performed most recently.
When the camera operation is performed within the prescribed time, that is, when the
predetermined time has not passed (in the case of No in step S002), the camera information
notification unit 320 carries out step S002 again.
[0084]
If the camera operation is not performed within the prescribed time, that is, if the predetermined
time has passed (in the case of Yes in step S002), the camera information notification unit
320 specifies the pan direction of the camera. The meeting terminal A20 is notified (step S003),
the distance between the camera and the subject is specified, and the meeting terminal A20 is
notified (step S004). The angle of view taken by the camera is specified and the meeting terminal
A20 is notified. (Step S005).
[0085]
As described above, in the present embodiment, the camera information notification unit 320
identifies the pan direction from the difference between the direction of the base (front) and the
direction of the optical axis of the camera, and the distance to the subject is predetermined. The
angle of view is determined according to the amount of extension of the lens unit, from the focal
length in a state in which the subject is in focus at the distance measurement point of.
[0086]
04-05-2019
22
When the camera state detection unit 210 of the conference terminal A20 receives the
notification in steps S003 to S005, the camera operation valid / invalid designation unit 240
issues a camera operation lock instruction to the camera A30 (step S006).
[0087]
When receiving the camera operation lock instruction in step S006, the camera operation valid /
invalid control unit 330 of the camera A30 sets the pan / zoom operation as invalid (step S007).
[0088]
The microphone array control unit 220 of the conference terminal A 20 performs microphone
array setting calculation processing described later (step S 008).
[0089]
Then, the microphone array control unit 220 instructs the microphone gain control unit 410 of
the microphone array A 40 to set a microphone gain including the setting value of the
microphone gain specified by the microphone array setting calculation process (step S 009).
[0090]
The microphone gain control unit 410 of the microphone array A 40 performs microphone gain
setting processing (step S010).
Specifically, the microphone gain control unit 410 sets a microphone gain for each of the
microphone amplifiers 430 a to 430 l in accordance with the microphone gain setting instruction
instructed in step S 009.
[0091]
The camera operation valid / invalid instruction unit 240 instructs the camera A 30 to release
the camera operation lock (step S011).
[0092]
04-05-2019
23
When receiving the camera operation lock release instruction in step S011, the camera operation
enable / disable control unit 330 of the camera A30 performs camera operation lock release
processing (step S012).
Specifically, the camera operation valid / invalid control unit 330 shifts to a state of effectively
accepting the pan / zoom operation.
[0093]
The above is the flow of the microphone array control process.
According to the microphone array control process, the distance between the position of the
image acquisition means capable of panning to the left and right and the position of the subject
reflected in the image acquired by the image acquisition means, and the direction of the pan of
the image acquisition means The distance between the position of the image acquisition means
and the position of the sound collection means capable of changing the sound collection range,
and the direction of the position of the sound collection means with reference to the position of
the image acquisition means The sound collection range can be changed.
That is, in the state where the camera and the microphone are arranged independently, it is
possible to improve the microphone sensitivity in the direction of the subject, so it is possible to
improve the quality of the sound collection.
[0094]
Next, processing of the microphone array setting calculation processing performed in step S008
of the above-mentioned microphone array control processing and its principle will be described
using FIG.
In the present description, it is assumed that the conference is conducted in the arrangement as
shown in FIG.
04-05-2019
24
That is, in the state where the camera A30 and the microphone array A40 shown in FIG. 5 are
arranged, the optical axis direction (pan angle 503) of the camera A30 directed to the speaker
and the direction of the microphone array A40 viewed from the camera A30. A difference
between (the installation direction 501) and the installation direction 501 is defined as a
difference angle α 504.
That is, the pan angle 503 from the front of the camera A30 is an angle obtained by combining
the installation direction 501 and the difference angle α504.
[0095]
The distance between the microphone array A40 and the subject is a distance a, the distance
between the subject and the camera A30 is a distance b502, and the distance between the
camera A30 and the microphone array A40 is c500. Further, it is assumed that the angle of view
505 taken by the camera A30 is an angle of view that allows the speaker to sufficiently capture
in the field of view.
[0096]
Further, the difference angle between the direction of the microphone array A40 viewed from the
speaker and the direction of the camera A30 is defined as a difference angle γ, and the
difference angle between the speaker direction viewed from the microphone array A40 and the
direction of the camera A30 Let be the difference angle β.
[0097]
In the case of arranging as described above, among the distance a, the distance b 502, the
distance c 500 and the difference angle α 504, the difference angle β, and the difference angle
γ, the relationship of cosine theorem 90 (the following formula 1, formula 2, formula 3) Is
established.
[0098]
<img class = "EMIRef" id = "205555372-00003" />
[0099]
<img class = "EMIRef" id = "205555372-00004" />
04-05-2019
25
[0100]
<img class = "EMIRef" id = "205555372-000005" />
[0101]
That is, it can be said that the above relationship is established.
Therefore, the microphone array control unit 220 specifies the direction in which the
microphone array A 40 collects the sound (that is, the difference angle β) by solving the
equations 1, 2, and 3 above.
[0102]
In the present embodiment, the microphone array control unit 220 receives the pan direction in
step S003, the distance from the camera A30 to the subject in step S004, and the information of
the angle of view in step S005, and the initialization process The distance 601 between the
camera A and the microphone array A of the microphone array position table 600 stored in the
above and the reference pan direction angle 602 can be read out.
The microphone array control unit 220 identifies the difference angle α 504 from the difference
between the pan direction and the reference pan direction angle 602, identifies the distance from
the camera A30 to the subject as the distance b 502, and determines the distance 601 between
the camera A and the microphone array A. Is specified as the distance c500, and is substituted
into each equation of the cosine theorem 90 to calculate the difference angle β.
[0103]
Then, the microphone array control unit 220 specifies the microphone 420 including the
direction of the difference angle β as the microphone in the sound collection direction, and also
specifies the microphone 420 adjacent to the microphone.
04-05-2019
26
Then, the output level (microphone gain setting value) of the microphone amplifier 430 is
specified using the information of the angle of view received in step S 005 and the information of
the microphone array output level table 650.
The sensitivity (gain) of the microphone in the sound collection direction is set higher than the
sensitivity of any other microphones, and the sensitivity (gain) of microphones other than the
microphones in the sound collection direction and the microphones adjacent to the microphones
in the sound collection direction By setting the sensitivity lower than that of any other
microphone, it is possible to obtain a high quality of utterance content of the speaker.
[0104]
The above is the principle and processing content of the microphone array setting calculation
processing.
[0105]
The video conference system 1 including the microphone control device to which the first
embodiment according to the present invention is applied has been described above.
According to the conference terminals A20, B21, and C22 which are the microphone control
devices to which the first embodiment is applied, the microphone sensitivity can be increased in
the direction of the subject in the state where the camera and the microphone are disposed
independently. It can be said that the quality of sound collection can be improved.
[0106]
The present invention is not limited to the above first embodiment.
The first embodiment can be variously modified within the scope of the technical idea of the
present invention. For example, in the first embodiment, the conference terminal A 20 specifies
the output level of the microphone amplifier in the sound collection direction according to the
angle of view. However, the present invention is not limited to this, and the output level may be
04-05-2019
27
adjusted according to the distance from the microphone array to the speaker. For example, the
output level adjustment table 700 shown in FIG. 11 may be stored in the microphone array
position data memory 230, and the output level calculated in the microphone array setting
calculation process may be adjusted. As a result, it is possible to set a more appropriate output
level, and hence it can be said that appropriate sound quality can be secured.
[0107]
Specifically, the output level adjustment table 700 stores the output level 702 of the microphone
selected as the sound collection direction in association with the distance 701 from the
microphone array to the speaker. When the distance from the microphone array A 40 to the
speaker, ie, the distance a, is calculated according to the cosine theorem 90, the microphone
array control unit 220 acquires the output level of the microphone according to the distance a
using the output level adjustment table 700. The microphone array control unit 220 sets the
output level within the range equal to or lower than the output level of the microphone in the
sound collection direction specified using the microphone array output level table 650 and the
angle of view. Then, the microphone array control unit 220 issues a microphone gain setting
instruction at the obtained output level.
[0108]
That is, according to the modified embodiment, it can be said that not only control of the sound
collection direction and the sound collection range of the microphone array A 40 but also
background noise reduction of the speaker can be performed.
[0109]
As an example, as shown in FIG. 12, it is assumed that the sound collection available range 520
of the microphone array A 40 is a sufficiently wide range with respect to the seating positions of
the conference participants.
In this case, the sound collection range of the microphone array A 40 after the sound collection
range control in the microphone control device according to the first embodiment is the sound
collection range 521 after the control. At this time, the sound collection range of the microphone
array A 40 is reduced to the sound collection range 522 by reducing the output level of the
microphone 420 selected as the sound collection direction using the table shown in FIG. That is,
04-05-2019
28
noise generated behind the speaker can be reduced.
[0110]
In the embodiment described above, the microphone array A40 is disposed on the conference
desk and the camera A30 is disposed on the conference console, but within the sound collection
range of the microphone array A40 and within the imaging range of the camera A30. In addition,
as long as the conference participants are at positions where the camera A30 can take a picture,
each may be placed anywhere. In addition, the functions of the respective units in the conference
terminal A20 described in the present embodiment may be dispersed and incorporated in the
camera A30 or the microphone array A40, and processing may be performed in cooperation with
the camera A30 and the microphone array A40. . Furthermore, the present invention can be
applied not only to video conferencing systems, but also to surveillance systems using network
surveillance cameras.
[0111]
Further, the variations of the above-described embodiment may be applied singly or in
combination.
[0112]
As described above, the present invention has been described focusing on the embodiment.
In the above embodiment, communication by the Internet etc. is assumed to be communication
by existing protocols such as HTTP, FTP, SIP, POP, SMTP, TCP / IP, etc. However, the present
invention is not limited to this, communication by other protocols It may be
[0113]
1 ... video conference system, 10 ... conference room A, 11 ... conference room B, 12 ... conference
room C, 20 ... conference terminal A, 21 ... conference terminal B, 22 ... ... conference terminal C,
30 ... camera A, 31 ... camera B, 32 ... camera C, 40 ... microphone array A, 41 ... microphone
array B, 42 ... microphone array C , 50: Display A, 51: Display B, 52: Display C, 60: Speaker A, 61:
Speaker B, 62: Speaker C, 70: Network, 80 ... server, 90 ... cosine law, 340 ... initial setting image,
350 ... scope, 420a-420l ... microphone, 430a-430l ... microphone amplifier, 500 ... distance c,
04-05-2019
29
501 ・ ・ ・ How to install , 502: distance b, 503: pan angle, 504: difference angle α, 505: angle
of view, 520: sound collection possible range, 521: sound collection range, 522 ··· Sound
collection range, 600: microphone array position table, 650: microphone array output level table,
700: output level adjustment table
04-05-2019
30
1/--страниц
Пожаловаться на содержимое документа