close

Вход

Забыли?

вход по аккаунту

JP2003008972

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2003008972
[0001]
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an
imaging device, an imaging method, an imaging device control system, and a storage medium.
[0002]
2. Description of the Related Art Conventionally, as shown in the block diagram of FIG. 12, an
imaging apparatus control system of this kind comprises an imaging means 1200 for imaging an
object and a sound source detection means for detecting a sound generated from the object 1209
and a sound source position estimating means 1208 for estimating a sound source position
based on an acoustic signal outputted from the sound source detecting means 1209, and an
automatic focusing based on estimated sound source position information outputted from the
sound source position estimating means 1208 A focusing means 1203 is provided to enable
quick focus control of the estimated sound source direction of the detected sound (see, for
example, Japanese Patent Application Laid-Open No. 12-101901).
[0003]
However, in such a conventional imaging device control system, the sound generated from the
subject in the peripheral portion of the imaging screen is the SN of the acoustic signal because of
the directivity of the sound source collecting means. There was a problem that the ratio
decreased.
04-05-2019
1
Moreover, in the conventional imaging control system, there existed a problem that an acoustic
abnormality was undetectable. Furthermore, when there are a plurality of cameras, there is a
problem that it is difficult to select an image of a camera in which an abnormality has occurred.
Furthermore, there is a problem that it is difficult to search the stored image data.
[0004]
The present invention has been made to solve such conventional problems, and provides an
imaging device control system capable of improving the S / N ratio of a picked up acoustic signal.
The present invention also provides an imaging device control system capable of detecting an
acoustic abnormality. Further, the present invention provides an imaging device control system
capable of easily selecting an image of a camera in which an abnormality has occurred when
there are a plurality of cameras. The present invention also provides an imaging device control
system that can easily search stored image data.
[0005]
SUMMARY OF THE INVENTION An image pickup apparatus according to the present invention
comprises an image pickup means for picking up an image of a subject, a sound input means for
inputting sound generated from the subject using a plurality of microphones, and the sound
input means A sound source direction estimating means for estimating a sound source direction
using an acoustic signal outputted by the sound source, and a directional sound collecting means
for picking up a sound with directivity in an estimated sound source direction outputted from the
sound source direction estimating means. It has the characteristic configuration. With this
configuration, it is possible to improve the SN ratio of the acoustic signal picked up while
imaging the direction in which the acoustic signal is generated.
[0006]
Further, the image pickup apparatus according to the present invention is characterized by
comprising an image pickup camera as the image pickup means and direction control means
capable of integrally moving the directions of the plurality of microphones as the sound input
means. have. With this configuration, it is possible to improve the SN ratio of the sound signal
collected without updating the coefficient of the directional sound collection unit.
04-05-2019
2
[0007]
Further, the direction control means of the image pickup apparatus of the present invention is
characterized in that the image pickup direction of the image pickup camera is directed to the
estimated sound source direction outputted from the sound source direction estimation means.
With this configuration, it is possible to improve the SN ratio of the sound signal collected
without updating the coefficient of the directional sound collection unit.
[0008]
Furthermore, the image pickup apparatus according to the present invention further comprises
an image extracting means for extracting a partial region of the image outputted from the image
pickup means based on the estimated sound source direction outputted from the sound source
direction estimation means. And a configuration characterized by With this configuration, it is
possible to extract an image near the sound source direction without moving the imaging
camera.
[0009]
Furthermore, the image pickup apparatus according to the present invention is characterized by
further comprising directional direction control means for sequentially updating the directional
sound collection direction based on the estimated sound source direction outputted from the
sound source direction estimation means. It has a configuration. With this configuration, it is
possible to improve the SN ratio of the collected sound signal.
[0010]
Further, the image pickup apparatus of the present invention is characterized in that a plurality
of microphones are installed around an image pickup camera lens which constitutes the image
pickup means. With this configuration, the imaging direction can be easily determined because
the estimated sound source direction output from the sound source direction estimation means
matches the imaging direction.
04-05-2019
3
[0011]
Further, the image pickup apparatus of the present invention has an arrangement characterized
in that it comprises an abnormal sound judging means for judging an abnormal sound using an
acoustic signal. With this configuration, acoustic abnormalities can be detected.
[0012]
Further, the image pickup apparatus of the present invention is characterized in that an
abnormal sound is determined using an acoustic signal outputted from the directivity collecting
means. With this configuration, the abnormal sound detection accuracy can be improved.
[0013]
In the image pickup apparatus control system according to the present invention, an image
pickup means for picking up an object, an acoustic input means for inputting a sound generated
from the object using a plurality of microphones, and an acoustic signal outputted from the
acoustic input means A sound source direction estimation unit that estimates a sound source
direction using a plurality of directional sound pickup units configured to pick up the sound with
directivity in the estimated sound source direction output from the sound source direction
estimation unit; An image pickup apparatus is provided, and the operation of the plurality of
image pickup apparatuses is controlled from a remote control apparatus installed at a remote
place. With this configuration, it is possible to improve the SN ratio of the acoustic signal picked
up while imaging the direction in which the acoustic signal is generated.
[0014]
In the image pickup apparatus control system according to the present invention, the image
pickup apparatus includes abnormal sound judging means, and the remote control apparatus
selects an image of an image pickup apparatus in which an abnormal sound is detected among a
plurality of image pickup apparatuses. And an image selection means for selecting an image.
With this configuration, when there are a plurality of imaging cameras, it is possible to easily
04-05-2019
4
select an image of the imaging camera in which an abnormality has occurred.
[0015]
Further, in the imaging control system according to the present invention, the remote control
device extracts a partial region of the image output from the imaging device based on the
estimated sound source direction output from the sound source direction estimating means. The
apparatus further comprises an image extracting unit. With this configuration, an image in the
vicinity of the sound source direction can be extracted without moving the imaging camera.
[0016]
Further, in the imaging control system according to the present invention, the remote control
device is characterized by comprising data storage means for storing an acoustic signal and an
abnormal sound discrimination result which is an output of the abnormal sound discrimination
processing in synchronization with a video. Have a configuration to This configuration makes it
easy to search for video data when an abnormal sound occurs.
[0017]
Further, in the image pickup control system according to the present invention, the data storage
means has a configuration characterized in that an acoustic signal output from the directional
sound collection means is stored. With this configuration, it is possible to make a search by
combining video data and sound data at the time of abnormal sound occurrence.
[0018]
In the image pickup apparatus control method of the present invention, an image pickup process
for picking up an object, an acoustic input process for inputting a sound generated from the
object using a plurality of microphones, and an acoustic signal outputted from the acoustic input
process. And a plurality of directional sound collecting processes for collecting sound with
directivity in the estimated sound source direction output from the sound source direction
estimation process. An imaging apparatus is provided, and this operation is controlled from a
04-05-2019
5
remote control device installed at a remote place. With this configuration, it is possible to
improve the SN ratio of the acoustic signal picked up while imaging the direction in which the
acoustic signal is generated.
[0019]
In the image pickup apparatus control method according to the present invention, the image
pickup apparatus includes an abnormal sound discrimination process, and the remote control
apparatus selects an image of an image pickup apparatus in which an abnormal sound is
detected among a plurality of image pickup apparatuses. And a video selection process. With this
configuration, when there are a plurality of imaging cameras, it is possible to easily select an
image of the imaging camera in which an abnormality has occurred.
[0020]
Furthermore, in the imaging device control method of the present invention, the remote control
device extracts a partial region in the image output from the imaging device based on the
estimated sound source direction output from the sound source direction estimation processing.
And image extraction processing. With this configuration, an image in the vicinity of the sound
source direction can be extracted without moving the imaging camera.
[0021]
Further, according to the image pickup apparatus control method of the present invention, the
remote control apparatus is provided with data accumulation processing for accumulating an
acoustic signal and an abnormal sound discrimination result which is an output of the abnormal
sound discrimination processing in synchronization with a video. It has the characteristic
configuration. This configuration makes it easy to search for video data when an abnormal sound
occurs.
[0022]
Further, in the image pickup apparatus control method of the present invention, the data
04-05-2019
6
accumulation process is characterized in that an acoustic signal output from the directional
sound collection process is accumulated. With this configuration, it is possible to make a search
by combining video data and sound data at the time of abnormal sound occurrence.
[0023]
Furthermore, the recording medium of the present invention has a configuration characterized
by storing a program for causing a computer to function as each means described in the abovementioned 1 to 13 so as to be callable from the computer. . With this configuration, it is possible
to improve the SN ratio of the acoustic signal picked up while imaging the direction in which the
acoustic signal is generated.
[0024]
Further, the recording medium of the present invention is characterized in that the steps
constituting the image pickup method according to the above-mentioned claims 14 to 18 are
readably stored by a computer. With this configuration, it is possible to improve the SN ratio of
the acoustic signal picked up while imaging the direction in which the acoustic signal is
generated.
[0025]
DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be
described below with reference to the drawings.
[0026]
As shown in FIG. 1, the image pickup apparatus according to the first embodiment of the present
invention comprises an image pickup camera 101, an image input unit 102, a CPU (Central
Processing Unit) 103, a sound source direction calculation unit 104, an external I / O. F 105,
image parameter control device 106, imaging direction control device 107, microphone array
108, sound input unit 109, ROM (Read Only Memory) 110, RAM (Random Access Memory) 111,
directivity sound collection unit 112 An apparatus 100 and an external terminal 113 are
provided.
[0027]
04-05-2019
7
The configuration of each block will be described below.
The imaging camera 101 includes a lens unit that performs focus adjustment and the like, an
aperture that adjusts the amount of light, and a CCD imaging device that converts an incident
optical image into an electrical signal.
The image input unit 102 is connected to the imaging camera 101, digitizes an image captured
by the imaging camera 101, and transfers the image to the RAM 111. The CPU 103 integrally
controls the entire imaging apparatus, reads a processing program stored in the ROM 110 into
the RAM 111, and controls various operations based on the processing program. The source
direction calculation unit 104 sequentially calculates the sound source direction based on the
output of the sound input unit 109. The external I / F 105 is connected to the external terminal
113, and outputs the image data and the directivity sound collection result calculated in the
directivity sound collection unit 112 described later to the external terminal 113. Also, a
command is received from the external terminal 113 to control imaging parameters and imaging
directions. The imaging parameters indicate focus, iris, gain, shutter speed, white balance, and
the like. The imaging parameter control device 106 is connected to the imaging camera 101 via a
control signal line, and can set imaging parameters. The imaging direction control device 107
controls the imaging direction of the imaging camera 101. The microphone array 108 is
configured by arranging a plurality of microphones. The output of the microphone array is used
for the calculation of the sound source direction by the sound source direction calculation unit
104 and the calculation of the directivity collection result of the directivity collection unit 112.
The sound input unit 109 is connected to the microphone array 108, amplifies the output signal
of the microphone array 108, converts it into digital data, and transfers it to the RAM 111. The
directivity collecting unit 112 is configured to collect directivity by giving directivity in a specific
direction using the output of the sound input unit 109, and to calculate a directivity collecting
result.
[0028]
FIG. 2 shows a flow chart of the first embodiment. First, in step S201, it is determined whether a
command has been sent from the external terminal 113 or not. If a command has been sent, the
process proceeds to step S203. If a command has not been sent, the process proceeds to step
S202. In step S202, the sound source direction is estimated. Next, even when there are a plurality
of sound sources, the operation of sound source direction estimation will be described using FIG.
04-05-2019
8
3 based on a method based on eigenvalue analysis with high direction detection accuracy. In step
S301, the output of the microphone array 108 composed of M microphones is periodically cut
out with a window having a window length W. At this time, the shape of the time window may be
rectangular, but it is more preferable that the amplitude at both ends of the time window is small,
such as a Hanning window. FIG. 4 shows a layout of the microphone array according to the first
embodiment of this invention. The microphone array 401 is attached to the imaging camera 402,
and the microphone array 401 and the imaging camera 402 can be integrally changed in
direction by the horizontal rotation table 404. The microphones do not necessarily have to be at
equal intervals, but they are equally spaced here because they simplify the calculation of the
direction control vector described later. In addition, the distance between the microphones needs
to be shorter than half the wavelength of the target sound source signal, and in that range, the
longer the length, the higher the estimation accuracy of the sound source direction. Also, the
number M of microphones is required to be (the number of sound sources assumed to be present
simultaneously + 1) or more, and the estimation accuracy of the sound source direction is
improved as the number M increases. In step S302, arithmetic averaging is performed on a
plurality of time windows with respect to the time signal cut out in step S301. The number of
time windows for averaging needs to be equal to or more than the number of sound sources
assumed to exist simultaneously. In step S303, the arithmetic mean result in step S302 is
converted into the frequency domain, and complex amplitudes are calculated for each frequency.
As a method of calculating the complex amplitude, a known method by FFT is suitable, but when
the number of frequencies to be calculated is 4 or less, a known method by DFT is suitable. In
step S304, a complex amplitude matrix is constructed using the complex amplitudes calculated in
step S303, and is represented as a column vector X [m] as in equation (1). Here, Xm (m = 1 to M)
is a complex amplitude at that frequency calculated from the m-th microphone input signal. Also,
the symbol T means a transposed matrix of the matrix [·].
[0029]
[Equation 1]
[0030]
In step S305, the correlation matrix is calculated using the complex amplitude matrix calculated
in step S304, and is represented by the matrix R [m, m] as in equation (2).
However, the symbol H means transpose complex conjugate.
04-05-2019
9
[0031]
[Equation 2]
[0032]
In step S306, the eigenvectors v1 [m], v2 [m] and vM [m] (m = 1 to M) of the correlation matrix R
are first calculated.
Since the matrix R is a Hermitian matrix, the eigenvectors can be calculated by a known QL
method after being converted to a tridiagonal matrix by a known Householder method. Next,
using the eigenvectors, a matrix Rn [m, m] corresponding to a noise component in the case where
there are K sound sources is calculated as in Expression (3). However, if the number of sound
sources K needs to be equal to or less than the number M of microphones, and if the number of
sound sources can not be estimated in advance, then K = M-1.
[0033]
[Equation 3]
[0034]
In step S307, the direction control vector d [m] is calculated as in equation (4).
However, τ [m] is defined by equation (5). C is the speed of sound. In addition, Δ [m] represents
the path difference, using the coordinates (x [m], y [m], z [m]) of the microphone, the direction
(α, β), and the distance L between the sound source and the microphone 1 It can be expressed
as equation (6). The path difference is calculated based on the distance between the sound
source and the microphone 1.
[0035]
[Equation 4]
[0036]
04-05-2019
10
[Equation 5]
[0037]
[Equation 6]
[0038]
In step S308, the directional power P [α, β] is calculated by changing α and β according to the
equation (7).
[0039]
[Equation 7]
[0040]
In step S309, the sound source direction is calculated using the directional power P [α, β]
calculated in step S308.
When the number of sound sources is 1, a combination of α and β which takes the maximum
value of P [α, β] is set as the sound source direction.
When the number of sound sources is plural, the number of local maximum values of P [α, β] is
the number of sound sources, and the combination of α and β corresponding to the respective
local maximum values is the sound source direction.
The estimated direction of the sound source can be calculated by the above processing.
In step S203, the camera direction is determined.
If an external command is detected in S201, the direction designated by the external command is
taken as the camera direction. If no external command is detected in S201, the camera direction
is determined using the sound source direction calculated in S309. When the number of sound
04-05-2019
11
sources is one, the sound source direction of the sound source is taken as the camera direction.
When the number of sound sources is more than one, the direction in which equation (7) is
maximum is taken as the camera direction. In step S204, the camera direction is directed to the
direction determined in step S204. In step S206, the directivity sound pickup result is calculated.
There are many directional sound collection methods in the literature "Small Special Issue
Microphone Array, Journal of the Acoustical Society 51, 384-414 (1995)" and the like.
[0041]
Here, based on the method of delay and sum array, the operation of the directional sound
collection will be described with reference to FIG. In step S501, a pointing direction is set. When
the array microphone and the camera body move integrally as in the present embodiment, the
pointing direction is set to a direction coincident with the imaging direction of the camera. In
step S502, the output signal delay of each of the microphones constituting the microphone array
108 is provided. The delay time sets a delay time for canceling the delay in each microphone
when the sound wave arrives from the pointing direction set in step S501. Assuming that the
delay time to be set is τ [m] (m = 1 to M), it can be expressed as the above-mentioned equation
(5). In step S503, the output signals of the microphones delayed in step S502 are added. The
directivity sound pickup result can be calculated by the above processing. In step S206, the
external I / F 105 transfers the image and the directivity collection result calculated in step S206
to the outside. In step S207, the termination condition is evaluated, and if the termination
condition is met, the processing is terminated. If the end condition is not met, the process returns
to step S201 to repeat the above-described processing.
[0042]
As described above, since the image pickup apparatus according to the first embodiment of the
present invention includes the sound source direction calculation unit 104, the directivity
collection unit 112, and the image pickup direction control unit 107, generation of acoustic
signals is performed. It is possible to improve the SN ratio of the acoustic signal collected without
updating the coefficient of the directional sound collection unit 112 while imaging the direction
in which the image is captured.
[0043]
As shown in FIG. 6, the image pickup apparatus according to the second embodiment of the
present invention comprises an image pickup camera 601, an image input unit 602, a CPU
04-05-2019
12
(central processing unit) 603, a sound source direction calculation unit 604, and an abnormal
sound determination. An external I / F 606, an image parameter control device 607, an image
extraction unit 608, a microphone array 609, an audio input unit 610, a read only memory
(ROM) 611, a random access memory (RAM) 612, a directivity sound collection unit 613 , And an
external terminal 613.
This is different from the first embodiment in that the abnormal sound judging unit 605, the
video extracting unit 608, and the secondary storage unit 614 are further provided in the
imaging device 600.
[0044]
The configuration of each block will be described below. The imaging camera 601 is composed of
a lens unit that performs focus adjustment and the like, an aperture that adjusts the amount of
light, and a CCD imaging device that converts an optical image incident thereon through these
into an electrical signal. The image input unit 602 is connected to the imaging camera 601,
digitizes the image captured by the imaging camera 601, and transfers the image to the RAM
612. The CPU 603 centrally controls the entire imaging apparatus, reads out a processing
program described later stored in the ROM 611 to the RAM 612, and controls various operations
based on the processing program. The sound source direction calculation unit 604 sequentially
calculates the sound source direction based on the output of the sound input unit 610. The
abnormal sound determination unit 605 detects an acoustic abnormality based on the output of
the acoustic input unit 610. The external I / F 606 is connected to the external terminal 615, and
outputs the image data and the directivity sound collection result calculated by the directivity
sound collection unit 613 described later to the external terminal 615. Also, a command is
received from the external terminal 615 to control imaging parameters and imaging directions.
The imaging parameters refer to focus, iris, gain, shutter speed, white balance, and the like. The
imaging parameter control device 607 is connected to the imaging camera 601 via a control
signal line, and can set imaging parameters. The video extraction unit 608 extracts an image
corresponding to the sound source direction estimated using the output of the sound source
direction calculation unit 604. The microphone array 609 is configured by arranging a plurality
of microphones. It is used for calculation of the sound source direction in the sound source
direction calculation unit 604 and calculation of the directivity sound collection result in the
directivity sound collection unit 613 described later. The sound input unit 610 is connected to
the microphone array 609, amplifies the output signal of the microphone array 609, digitizes it,
and transfers it to the RAM 612. The directivity collecting unit 613 uses the output of the sound
input unit 610 to collect directivity by giving directivity in a specific direction, and calculates a
directivity collecting result. The secondary storage device 614 uses a hard disk, a CD-ROM, etc.,
04-05-2019
13
and stores information such as image data and a directivity sound collection calculation result in
the secondary storage device 614.
[0045]
FIG. 7 shows a flow chart of the second embodiment. First, in step S701, acoustic variation is
detected. As a variation detection method, there is a simple method of monitoring the output of
one of the microphone arrays 609 and assuming that an acoustic variation is detected when the
voltage of the output exceeds a predetermined threshold. As another variation detection method,
an average level of the output of one microphone in the microphone array 609 in a
predetermined time length is calculated, and an acoustic variation is detected when a
predetermined level difference from the average level is exceeded How to do it is desirable. In
step S702, the sound source direction is estimated. The method of estimating the sound source
direction is the same as step S202 in the first embodiment. In step S703, the image extraction
direction is determined using the sound source direction calculated in step S702. When the
number of sound sources is 1, the sound source direction of the sound source is taken as the
image extraction direction. When the number of sound sources is more than one, the direction in
which equation (7) is maximized is taken as the image extraction direction. In step S704, video
extraction is performed. Among the images captured at a wide angle, a partial region centered on
the image extraction direction determined in step S703 is extracted. By this processing, it is
possible to extract an image in the vicinity of the sound source without moving the imaging
camera. In step S 705, the directivity sound collection result is calculated. The method of
calculating the directional sound collection result is the same as step S206 in the first
embodiment. In step S706, abnormal sound determination is performed. Abnormal sound
determination is to detect an acoustic abnormality and classify the sound using an output signal
from the microphone array 509.
[0046]
FIG. 8 is a flowchart of the abnormal sound determination according to the second embodiment.
In step S801, the band level is calculated using the directivity collection result in step S705. As
the calculation method of the band level, it is suitable to calculate the band level after converting
to the frequency domain by known FFT. The bandwidth is suitably 1/3 octave bandwidth, and the
number of bands N is suitably 25 bands of 50 Hz to 12.5 kHz. The band level calculated
individually is represented as Li (i = 1 to N). In step S802, the band level L of all bands is
calculated according to equation (8).
04-05-2019
14
[0047]
[Equation 8]
[0048]
In step S803, coordinates on the discrimination function Xj (J = 1 to M) are calculated according
to equation (9).
The number of dimensions M is (the number of discrimination groups-1), and the discrimination
function coefficient α ij is calculated in advance by known discrimination analysis.
[0049]
[Equation 9]
[0050]
In step S804, a square distance Dk between the on-discrimination function coordinates calculated
in step S803 and the center coordinates Yk of the respective determination groups is calculated
according to equation (10).
The central coordinates Yk of each discrimination group are calculated in advance by known
discriminant analysis.
[0051]
[Equation 10]
[0052]
At step 805, k that determines the minimum square distance Dk calculated at step S804 is
determined as a discrimination group.
04-05-2019
15
Abnormal sound determination is performed by the above processing. In step S 707, the
directivity sound collection result in step S 705 and the abnormal sound determination result in
step S 706 are stored in association with the image of the imaging camera. A hard disk or the like
is suitable as the storage medium. In step S708, the directivity sound collection result in step
S705 and the abnormal sound determination result in step S706 are transferred to the outside
via the external I / F 506 in association with the image of the imaging camera. In step S 708, the
end condition is evaluated, and if the end condition is met, the process ends. If the end condition
is not met, the process returns to step S701 to repeat the above-described processing.
[0053]
Although the arrangement of the microphones may be the one described in the first embodiment
of FIG. 4, the second embodiment is different in that the arrangement of FIG. 9 is used. With the
configuration in which the microphone array 901 is disposed around the lens 903, an effect can
also be obtained for sound source direction estimation. The number of microphones needs to be
three or more, and six to eight are appropriate.
[0054]
As described above, the imaging apparatus according to the second embodiment of the present
invention is provided with the abnormal sound judging unit 605 and the video extracting unit
608, so that an acoustic abnormality is detected and the image in that direction is detected. It is
possible to improve the SN ratio of the sound signal collected while extracting the sound signal.
[0055]
As shown in FIG. 10, the image pickup apparatus according to the third embodiment of the
present invention includes image pickup cameras 1009a to 1009d, an external I / F 1001, a CPU
(Central Processing Unit) 1002, an image selection unit 1003, image display A section 1004, a
control panel 1005, a ROM (Read Only Memory) 1006, a RAM (Random Access Memory) 1007,
and a secondary storage device 1008 are provided.
This is different from the first embodiment in that the abnormal sound judging unit 605, the
video extracting unit 608, and the secondary storage unit 614 are further provided in the
04-05-2019
16
imaging device 600.
[0056]
The configuration of each block will be described below. The imaging camera 601 is composed of
a lens unit that performs focus adjustment and the like, an aperture that adjusts the amount of
light, and a CCD imaging device that converts an optical image incident thereon through these
into an electrical signal. The image input unit 602 is connected to the imaging camera 601,
digitizes the image captured by the imaging camera 601, and transfers the image to the RAM
612. The CPU 603 centrally controls the entire imaging apparatus, reads out a processing
program described later stored in the ROM 611 to the RAM 612, and controls various operations
based on the processing program. The sound source direction calculation unit 604 sequentially
calculates the sound source direction based on the output of the sound input unit 610. The
abnormal sound determination unit 605 detects an acoustic abnormality based on the output of
the acoustic input unit 610. The external I / F 606 is connected to the external terminal 615, and
outputs the image data and the directivity sound collection result calculated by the directivity
sound collection unit 613 described later to the external terminal 615. Also, a command is
received from the external terminal 615 to control imaging parameters and imaging directions.
The imaging parameters refer to focus, iris, gain, shutter speed, white balance, and the like. The
imaging parameter control device 607 is connected to the imaging camera 601 via a control
signal line, and can set imaging parameters. The video extraction unit 608 extracts an image
corresponding to the sound source direction estimated using the output of the sound source
direction calculation unit 604. The microphone array 609 is configured by arranging a plurality
of microphones. It is used for calculation of the sound source direction in the sound source
direction calculation unit 604 and calculation of the directivity sound collection result in the
directivity sound collection unit 613 described later. The sound input unit 610 is connected to
the microphone array 609, amplifies the output signal of the microphone array 609, digitizes it,
and transfers it to the RAM 612. The directivity collecting unit 613 uses the output of the sound
input unit 610 to collect directivity by giving directivity in a specific direction, and calculates a
directivity collecting result. As the secondary storage device 614, a hard disk, a CD-ROM or the
like is used. The CPU 1002 generally controls the entire imaging apparatus, reads a processing
program described later stored in the ROM 1005 to the RAM 1006, and controls various
operations based on the processing program. The image selection unit 1003 selects the images
of the plurality of imaging cameras 1009a to 1009d based on the abnormal sound determination
results of the imaging cameras 1009a to 1009d or the command input from the control panel
1005.
The video display unit 1004 displays the video selected by the video selection unit 1003. The
04-05-2019
17
control panel 1005 inputs commands such as camera switching and imaging direction control.
The secondary storage device 1008 uses a hard disk, a CD-ROM, etc., and stores information
such as image data and a directivity sound collection calculation result in the secondary storage
device 614.
[0057]
The operation of the image pickup apparatus control system configured as described above will
be described with reference to FIG. In step S1101, it is detected whether a command has been
input from the control panel 1005. If a command is detected, the process proceeds to the
corresponding step S1102 to step S1105. If no command is detected, the process advances to
step S1107. In step S1102, one or more of the imaging cameras 1009a to 1009d are selected
and displayed on the screen of the video display unit 1104. In step S1103, the directions of the
imaging cameras 1009a to 1009d selected in step S1102 are controlled. When the imaging
camera includes the rotating table as shown in FIG. 4, the imaging direction of the camera is
directed to the designated direction. When the imaging camera does not include the rotation
table as shown in FIG. 9, a partial area centered on the specified direction is extracted and
displayed on the screen of the video display unit 1104. In step S1104, in the case where each of
the imaging cameras 1009a to 1009d does not include the directivity collecting unit, the
directivity collecting result is calculated using the input signal of the microphone array, and is
output as a sound. In addition, the calculation method of the directivity sound-collection result
follows step S206 of the said 1st embodiment. Further, in the case where each of the imaging
cameras 1009 a to 1009 d includes the directivity collecting unit, the directivity collecting result
calculated by the designated imaging camera is received and output as a sound. In step S1105,
the directivity collection result in step S1104 is stored in the secondary storage device 1008 in
association with the image of the imaging camera. A hard disk or the like is suitable as the
storage medium. In step S1107, it is detected in each of the imaging cameras 1009a to 1009d
whether an abnormal sound is detected. If no abnormal sound is detected, the process returns to
step S1101. If an abnormal sound is detected, the process proceeds to step S1109. In step 1109,
in association with the image of the imaging camera in which the abnormal sound is detected, the
sound in the imaging camera in which the abnormal sound is detected and the abnormal sound
discrimination result are stored together. A hard disk or the like is suitable as the storage
medium. Thereafter, the process returns to step S1101. In step S1106, the end condition is
evaluated, and if the end condition is met, the process ends. If the end condition is not met, the
process returns to step S1101 to repeat the above-described processing.
[0058]
04-05-2019
18
As described above, the imaging device control system according to the third embodiment of the
present invention is different from the first and second embodiments in that an image selection
unit 1003 is further provided. According to this configuration, even in the case where there are a
plurality of imaging cameras 1009a to 1009d, it is possible to detect an acoustic abnormality
and to select and display an image of the corresponding imaging camera.
[0059]
As described above, according to the present invention, the image pickup means for picking up
an object, the sound input means for inputting the sound generated from the object using a
plurality of microphones, and the sound input means A sound source direction estimation unit
that estimates a sound source direction using an acoustic signal output from the sound source;
and a directional sound collection unit that picks up the directivity of the estimated sound source
direction output from the sound source direction estimation unit. It is possible to provide an
imaging device control system having an excellent effect of being able to improve the SN ratio of
an acoustic signal picked up while imaging the direction in which the acoustic signal is
generated.
[0060]
Brief description of the drawings
[0061]
1 is a block diagram showing an imaging apparatus according to the first embodiment of the
present invention.
[0062]
2 is a flow chart showing the operation of the imaging device according to the first embodiment
of the present invention
[0063]
Flow chart showing the operation of the sound source direction estimation process of the
imaging device of the first embodiment of the present invention
[0064]
4 is a block diagram showing an imaging apparatus according to the first embodiment of the
04-05-2019
19
present invention
[0065]
Flow chart showing the operation of the directional sound collection result calculation process of
the imaging device according to the first embodiment of the present invention
[0066]
6 is a block diagram showing an image pickup apparatus according to the second embodiment of
the present invention.
[0067]
7 is a flow chart showing an imaging apparatus according to the second embodiment of the
present invention.
[0068]
Flow chart showing the operation of the abnormal sound detection process of the imaging device
of the second embodiment of the present invention
[0069]
9 is a block diagram showing an imaging apparatus according to the second embodiment of the
present invention.
[0070]
10 is a block diagram showing an imaging apparatus control system according to the third
embodiment of the present invention.
[0071]
Flow chart showing the operation of the imaging device control system of the third embodiment
of the present invention
[0072]
12 is a block diagram showing a conventional imaging device control system
04-05-2019
20
[0073]
Explanation of sign
[0074]
100 imaging apparatus 101 imaging camera 102 image input unit 103 CPU 104 sound source
direction calculation unit 105 external I / F 106 imaging parameter control device 107 imaging
direction control device 108 microphone array 109 acoustic input unit 110 ROM 111 RAM 112
directivity sound collection unit 113 external terminal 401 microphone Array 402 Imaging
camera 403 Lens 404 Horizontal rotation table 600 Imaging device 601 Imaging camera 602
Image input unit 603 CPU 604 Sound source direction calculation unit 605 Abnormal sound
discrimination unit 606 External I / F 607 Imaging parameter control device 608 Video
extraction unit 609 Microphone array 610 Sound Input unit 611 ROM 612 RAM 613 Directional
sound collection unit 614 Secondary storage unit 615 External terminal 901 Microphone array
902 Imaging camera 903 Lens 1000 Control device 1001 External I / F 1002 CPU 1003 Video
selection unit 1004 Video display unit 1005 Control panel 1006 ROM 1007 RAM 1008
Secondary storage device 1009a Imaging camera 1009b Imaging camera 1009c Imaging camera
1009d Imaging camera
04-05-2019
21
1/--страниц
Пожаловаться на содержимое документа