close

Вход

Забыли?

вход по аккаунту

JP2013106298

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2013106298
Abstract: The present invention is presented to a user so that the situation of the presence of an
audio signal generation source during imaging can be understood at first glance. A display unit,
such as a touch panel, for displaying an acquired image, an audio direction detection unit for
separating an acquired audio into an audio signal for each audio signal generation source and
detecting a direction for each audio signal generation source, A graphic or character, for
example, an icon is generated corresponding to the audio signal generation source detected by
the audio direction detection unit. The control unit is configured to convert the detected direction
into a position on the display area of the touch panel and display an icon in the vicinity of the
position. [Selected figure] Figure 1
Imaging control apparatus, imaging control method, program for imaging control method, and
imaging apparatus
[0001]
The present disclosure relates to an imaging control apparatus, an imaging control method, an
imaging control method program, and an imaging apparatus applied to an imaging apparatus
that records, for example, a moving image signal and an audio signal on a recording medium.
[0002]
For example, if the sound source of the imaging target can be displayed on the display unit while
capturing a moving image by the imaging device, the sound level of the desired sound source can
be adjusted, and the adjusted sound can be recorded with the image on the recording medium.
03-05-2019
1
As a result, unnecessary sound can be removed or reproduction with suppressed sound level can
be performed. In particular, there is a demand for recording except for sounds other than a
person as an object or a sound generated by an object (a tonic sound) or a sound that disturbs a
tonic sound. Since the determination as to whether the sound is unnecessary is different
depending on the photographer, it is preferable that the ambient sound be visualized at the time
of imaging and the photographer manipulate the ambient sound.
[0003]
For example, Patent Document 1 describes a visualization technique for displaying an audio
signal generation source on a display device. In Patent Document 1, the position (direction) of a
sound source is estimated from the difference in time for sound at a specific frequency to be
measured to reach a plurality of microphones, and the estimated position of the sound source is
displayed.
[0004]
According to Patent Document 2, when capturing a moving image of a digital camera, recording
is performed by a microphone array including a plurality of microphones, a two-dimensional
image having a color corresponding to a sound pressure level is generated, and a twodimensional image is displayed on a display unit. It is described. Furthermore, Patent Document
2 describes suppression or enhancement of a sound source in the direction of the sound source.
[0005]
Patent Document 1: Japanese Patent Application Laid-Open No. 2003-111183 Patent No.
453985
[0006]
In Patent Document 1, the position of an audio signal generation source such as noise is
displayed on the screen, and basically, the position of sound of one type of frequency is
displayed.
03-05-2019
2
Therefore, no consideration is given to the point of displaying a plurality of audio signal
generation sources present at the time of imaging.
[0007]
According to Patent Document 2, a contour line shaped two-dimensional image having a color
corresponding to a sound pressure level is superimposed on a captured image (through image).
This method is suitable for grasping the distribution of sound, but it is difficult to confirm at a
glance the presence of sounds generated by individual persons and objects during imaging.
Furthermore, only the audio signal generation source within the range of the captured image is
visualized, and the sound from the outside of the range of the captured image and the wind noise
generated by direct impact of the wind against the microphone can not be displayed. there were.
Furthermore, when an operation such as level emphasis or suppression is performed on a
visualized audio signal generation source by a touch panel, it is difficult to know which place in
the display area may be touched. As a result, there has been a problem that it is not possible to
manipulate the sound of the intended audio signal generation source or it is difficult to confirm
the result of the manipulation.
[0008]
Therefore, the present disclosure solves these problems, and can present an imaging control
apparatus, an imaging control method, and an imaging control method that can present the
situation of the presence of the audio signal generation source during imaging at a glance. And
providing an imaging device.
[0009]
In order to solve the above-mentioned problems, the present disclosure separates the acquired
audio into an audio signal for each audio signal generation source, and detects the direction for
each audio signal generation source. A voice direction detection unit and a voice signal
generation source detected by the voice direction detection unit to generate a figure or a
character, convert the detected direction into a position on the display area of the display unit,
and And a control unit for displaying a figure or a character in the vicinity.
[0010]
The present disclosure displays an acquired image on a display unit, separates the acquired audio
03-05-2019
3
into an audio signal for each audio signal generation source by an audio direction detection unit,
detects a direction for each audio signal generation source, and controls The unit generates a
figure or character corresponding to the voice signal generation source detected by the voice
direction detection unit, converts the detected direction into a position on the display area of the
display unit, and This is an imaging control method for displaying characters.
[0011]
According to the present disclosure, a computer displays an acquired image on a display unit,
separates the acquired audio into an audio signal for each audio signal generation source by an
audio direction detection unit, and determines the direction for each audio signal generation
source. The control unit generates a figure or character corresponding to the audio signal
generation source detected by the audio direction detection unit, converts the detected direction
to a position on the display area of the display unit, and Is a program for executing an imaging
control method for displaying graphics or characters in the vicinity of.
[0012]
The present disclosure relates to an imaging unit for acquiring an image, a plurality of
microphones for acquiring audio, a display unit for displaying an image acquired by the imaging
unit, and an audio signal of audio acquired by the microphone for each audio signal generation
source. A voice direction detection unit that detects the direction of each voice signal generation
source, and generates a figure or character corresponding to the voice signal generation source
detected by the voice direction detection unit, and displays the detected direction as a display
unit Control unit for converting to a position on the display area of the screen, and displaying a
figure or a character in the vicinity of the position, and manipulating the figure or the character
to remove voice generated by the voice signal generation source corresponding to the figure or
the character And an audio operation unit for suppressing or emphasizing, and a recording unit
for recording the acquired image data and the audio data processed by the audio operation unit
on a recording medium. .
[0013]
The present disclosure separates a voice of a predetermined time acquired together with a still
image into voice signals for each voice signal generation source, and detects a direction of each
voice signal generation source, a still image, voice signal generation A recording unit for
recording an audio signal for each source and information on the direction for each audio signal
generation source on a recording medium; a reproduction unit for reproducing a still image from
the recording medium, an audio signal for each audio signal generation source, and direction
information A display unit for displaying a still image reproduced from the recording medium by
the reproduction unit, and a graphic or character corresponding to an audio signal for each audio
03-05-2019
4
signal generation source reproduced from the recording medium by the reproduction unit, and
reproduced It is an imaging control device provided with a control unit which converts
information of the determined direction into a position on a display area of the display unit and
displays a figure or a character in the vicinity of the position.
[0014]
The present disclosure separates a voice of a predetermined time acquired together with a still
image into voice signals for each voice signal generation source, and detects a direction of each
voice signal generation source, a still image, voice signal generation A recording unit for
recording on a recording medium an audio signal for each source and position information
obtained by converting information on a direction of each audio signal generation source into a
position on a display area of a display unit; A display unit for displaying a still image, and a
graphic or character corresponding to an audio signal for each audio signal generation source
reproduced from the recording medium by the reproduction unit, and in the vicinity of the
position designated by the reproduced position information It is an imaging control apparatus
provided with the control part which displays a figure or a character.
[0015]
The present disclosure superimposes and displays an icon corresponding to the audio signal
generation source being imaged on the captured image, so it is easy to understand at a glance
what kind of sound is present during imaging.
Therefore, images and sounds can be recorded by removing, removing or emphasizing unwanted
sounds at the time of imaging.
Furthermore, a still image with sound can be created, and an operation on the sound can be
performed at the time of reproduction.
[0016]
It is a block diagram showing composition of a 1st embodiment of this indication.
03-05-2019
5
It is a basic diagram used for explanation of detection of a sound source direction.
It is a flowchart used for operation ¦ movement description of 1st Embodiment of this indication.
It is a basic diagram used for explanation of the 1st example of icon display of a 1st embodiment
of this indication.
It is a basic diagram used for explanation of the 2nd example of icon display of a 1st embodiment
of this indication.
It is a basic diagram used for explanation of the character string display of a 1st embodiment of
this indication. It is a basic diagram used for explanation of the 3rd example of icon display of a
1st embodiment of this indication. It is a basic diagram used for explanation of icon operation in
a 1st embodiment of this indication. It is a block diagram showing composition of a 2nd
embodiment of this indication. It is a basic diagram used for explanation of icon operation in a
2nd embodiment of this indication.
[0017]
The embodiments described below are preferable specific examples of the present disclosure,
and various technically preferable limitations are attached. However, the scope of the present
disclosure is particularly described in the following description. It is not limited to these
embodiments as long as there is no.
[0018]
First Embodiment "Configuration of Entire Imaging Device" A first embodiment of the present
disclosure will be described.
An example of an imaging apparatus to which the present disclosure can be applied will be
described with reference to FIG. The imaging apparatus includes an optical block 1, a camera unit
2, and an image processing apparatus 3 as components related to image processing.
03-05-2019
6
[0019]
The optical block 1 includes a lens, a focusing mechanism, a shutter mechanism, an iris (iris)
mechanism, and the like. The camera unit 2 includes an imaging element such as a charge
coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), a preprocessing
circuit, a driver, a timing generation circuit, and the like.
[0020]
The image sensor photoelectrically converts the subject image from the optical block 1 and
outputs the image to the preprocessing circuit. The preprocessing circuit performs CDS
(Correlated Double Sampling) processing on the supplied imaging signal to improve the S / N
ratio, performs AGC (Automatic Gain Control) processing, and controls the gain, and , A / D
(Analog / Digital) conversion to form imaging data converted into a digital signal.
[0021]
Digital imaging data from the camera unit 2 is supplied to the video processing device 3. The
image processing device 3 performs camera signal processing such as AF (Auto Focus), AE (Auto
Exposure), AWB (Auto White Balance), or the like on the imaging data. The image data subjected
to camera signal processing is data compressed by a predetermined compression method and
supplied to the control unit 4. The audio data processed as described later is supplied to the
control unit 4.
[0022]
The control unit 4 is a microcomputer configured by a program 43 stored in a CPU (Central
Processing Unit) 41, a memory 42 such as a RAM (Random Access Memory), a ROM (Read Only
Memory), etc. Control each part.
[0023]
A storage device 5 as a data recording medium is connected to the control unit 4.
03-05-2019
7
The storage device 5 is a hard disk, a memory card or the like. In the control unit 4, imaging data
(image data and audio data) having a predetermined data structure is recorded in the storage
device 5.
[0024]
Furthermore, a touch operation compatible liquid crystal display unit (hereinafter referred to as a
touch panel as appropriate) 6 is connected to the control unit 4. A display signal from the control
unit 4 is supplied to the touch panel 6. The touch panel 6 displays an image at the time of
imaging (hereinafter, appropriately referred to as a through image) and also displays a
reproduced image from the storage device 5. An operation signal generated corresponding to the
operation of the touch panel 6 is supplied to the control unit 4. As described later, the icon
display signal generated in the icon generation device 17 and generated in the icon display
device 18 is supplied to the touch panel 6 and displayed on the touch panel 6.
[0025]
At the time of photographing, the control unit 4 controls the focus mechanism, the shutter
mechanism, and the aperture mechanism of the optical block 1 to capture an object image and
provide the object image to the imaging device. Furthermore, the lens apparatus of the optical
block 1 is made exchangeable. For example, a microcomputer is configured inside the lens
apparatus, and information such as the type of the lens apparatus and the current focal length is
transmitted to the control unit 4.
[0026]
The recorded data recorded is read from the storage device 5 in response to an operation input
from the user and supplied to the control unit 4. The control unit 4 performs decompression
processing (decompression processing) of the compressed data. The decompressed image data is
supplied to the touch panel 6 through the display controller as necessary. Thereby, an image
according to the reproduction data is displayed on the screen of the touch panel 6.
[0027]
03-05-2019
8
By the above-described imaging device, an image (still image or moving image) of a subject can
be photographed and recorded in the storage device 5. Furthermore, it is possible to read out the
image data recorded in the storage device 5, display the image on the touch panel 6, and
arbitrarily view and edit.
[0028]
Here, an operation at the time of moving image capturing of the above-described imaging device
will be schematically described. A digital imaging signal from the camera unit 2 is supplied to the
video processing device 3. In the video processing device 3, the image data is subjected to image
quality correction processing, and is supplied to the control unit 4 as image data of a through
image. The image data output from the control unit 4 is displayed on the touch panel 6 as a
through image. It becomes possible to perform angle of view alignment while viewing the
through image displayed on the touch panel 6.
[0029]
Then, when a button on the touch panel 6 or a mechanical recording button is pressed, the
recording operation is started, and the recording image data processed by the video processing
device 3 is supplied to the control unit 4. In the control unit 4, processing such as compression
coding is performed to generate recording data, and the recording data is stored in the storage
device 5.
[0030]
On the other hand, when the data stored in the storage device 5 is reproduced, the recording
data is supplied to the control unit 4 and is decoded by the control unit 4. The decoded image
data is supplied from the control unit 4 to the touch panel 6, and the reproduced image is
displayed on the touch panel 6.
[0031]
03-05-2019
9
Processing of Audio Signal Audio processing of the above-described imaging device will be
described. In the first embodiment of the present disclosure, a microphone array 11 including a
plurality of microphones (sound sensors) M1, M2, ..., Mn is used. For example, n / 2 microphones
arranged at predetermined intervals in the horizontal direction and n / 2 microphones arranged
at predetermined intervals in the vertical direction are made to collect sound in front of the
imaging device. Output signals of the microphones of the microphone array 11 are supplied to an
amplifier unit 12 including amplifiers A1 to An. The output signals of the respective amplifiers
are respectively supplied to A / D converters AD1 to ADn of the digitizing circuit 13 and
converted into digital audio signals.
[0032]
The digital audio signal from the digitizing circuit 13 is supplied to the sound source direction
detection device 14. The sound source direction detection device 14 estimates the sound source
direction corresponding to the audio signal acquired by the microphone array 11, and detects the
size of each sound source. A known method can be used as a sound source direction detection
method. That is, it is possible to use a method of estimating a direction by detecting a time
difference of signals received by each microphone, a method of obtaining as a spatial frequency
using a matrix, or the like. Furthermore, a phase difference spectrum and a power spectrum of
sound signals of two channels obtained by two microphones arranged at predetermined intervals
may be obtained, and a direction of each sound source may be obtained based on these.
Generally, at least two microphones are required to estimate the sound source direction.
[0033]
FIG. 2A schematically illustrates detection of the direction of a plurality of, for example, three
sound sources. In a horizontal plane in front of the imaging device, the sound source P1 has a
direction θx1 (an angle with respect to the front direction of the imaging device), the sound
source P2 has a direction θx2, and the sound source P3 has a direction −θx3. Furthermore, in
the front vertical plane of the imaging device, the sound source P1 has a direction θy1 (an angle
with respect to the front direction of the imaging device), the sound source P2 has a direction
θy2, and the sound source P3 has a direction −θy3.
[0034]
03-05-2019
10
The sound source direction detection device 14 estimates the direction and magnitude of each
sound source, and outputs information on the direction and magnitude to the control device 4. As
shown in FIG. 2B, the control device 4 converts direction information into position information
(xy coordinates) on a display area of the touch panel 6 as a display device (that is, a display area
of a through image or a reproduced image). However, when the information on the position of
the converted result is out of the display area of the touch panel 6, instead of the coordinates,
information indicating that, for example, data indicating out of area is set. In the example of
FIG. 2B, the sound source P1 is located at (x1, y1), the sound source P2 is located at (x2, y2), and
the sound source P3 is located outside the display area.
[0035]
Furthermore, even if it is out of the area, information may be set that indicates on which side of
the display area on the upper, lower, left, and right sides it is present. For example, when the
sound source is present on the right side outside the area, data indicating right outside the
area may be set. The reason for converting the direction information into the position
information (xy coordinates) on the display area of the touch panel 6 is to display an icon
corresponding to the sound source near the position of the sound source. Information on the
direction and size of each sound source is stored in the memory of the control device 4. The
detection of the direction and the size of the sound source stored in the memory of the control
device 4 is always performed, for example, at predetermined time intervals during imaging
standby and imaging, and the information in the memory is updated.
[0036]
The n-channel digital audio signal received by the microphones M1 to Mn is supplied from the
sound source direction detection device 14 to the sound source separation device 15. The sound
source separation device 15 separates and outputs each audio signal in the direction estimated
by the sound source direction detection device 14. The speech signal separation can be
performed using known methods such as a method of separating a speech signal using direction
estimation results and a method of separating a speech signal using independent component
analysis. At the output of the sound source separation device 15, an audio signal of a sound
source whose direction can not be specified is also extracted.
[0037]
03-05-2019
11
An audio signal output from the sound source separation device 15 is supplied to the control unit
4. In the control unit 4, characteristic sound data of the separated audio signal is stored in the
memory. The control unit 4 stores typical sound data. The icon creation device 17 compares the
sound data characterizing the individual audio signals stored in the memory in the control unit 4
with the representative sound data. The type of the separated audio signal is determined based
on the presence / absence (magnitude) of the correlation, and the icon corresponding to the
determination result is assigned to the separated audio signal.
[0038]
As an example, as shown in FIG. 2C, when the sound source P1 is determined to be a human
voice, an icon imitating a human face is assigned to the sound source P1. Similarly, when the
sound source P2 is determined to be a wave sound, an icon imitating a wave is assigned to the
sound source P2. Furthermore, in the case of a sound source with few features, such as wind
noise, the type of the sound source can not be determined, and a predetermined icon indicating
that the sound source can not be determined is assigned. Furthermore, the size of the icon to be
assigned is approximately proportional to the detected size (signal level) of the sound source. The
magnitude of the sound source may be determined based on the absolute signal level or may be
determined based on the relative magnitudes of multiple sound sources present in the same
image.
[0039]
An icon corresponding to the type and size of the sound source created by the icon creation
device 17 is supplied to the icon display device 18. The icon display device 18 displays the
created icon on the display area of the touch panel 6. In this case, an icon is displayed in the
vicinity of the position obtained from the sound source direction. The user can easily confirm the
sound around the imaging device from the position, size, type, state, etc. of the icon displayed on
the touch panel 6. The audio removal device 16 is configured to mute the audio signal
corresponding to the sound source for which the icon removal operation has been performed,
and outputs the audio signal after the removal processing of the audio signal, and the video
signal from the video processing device 3 And the control unit 4.
[0040]
03-05-2019
12
In the speech removal apparatus 16, processing for suppressing or emphasizing the speech
signal may be performed. In that case, the voice elimination device 16 is configured by a variable
gain amplifier whose gain is controlled by the control signal from the control unit 4. The
recording processing unit in the control unit 4 generates a recording signal, and the recording
signal is recorded on the storage device 5.
[0041]
Furthermore, the icon display device 18 is controlled by the control unit 4 so that the operation
mode relating to the display of the icon can be switched. The first operation mode is an off mode.
In the off mode, the icon is not displayed. The second operation mode is a full display mode. In
the all display mode, icons corresponding to all detected sound sources are displayed. The third
operation mode is a partial display mode. This is a mode for displaying the icons of a
predetermined number of sound sources at a high level with a large level. The fourth operation
mode is a user setting mode. In this mode, the user sets an icon to be displayed.
[0042]
FIG. 3 is a flowchart for explaining the flow of audio signal processing. In the first step S1, the
microphone array 11 picks up the sound generated during imaging. In step S2, the direction and
the size of the sound source of the collected sound signal are detected by the sound source
direction detection device 14. Furthermore, as described above, the direction of each sound
source is converted into the information of the position on the display area of the touch panel 6.
[0043]
In step S3, the sound source separation device 15 separates the sound signal of the sound source
in each direction (including the sound source whose direction can not be identified). The
separated audio signal is stored in the memory of the control unit 4 (step S4).
[0044]
03-05-2019
13
In step S5, icon creation processing by the icon creation device 17 and icon display processing
by the icon display device 18 are performed. As described above, the data of a plurality of
representative sounds stored in the control unit 4 is compared with the data of the audio signal
of the sound source in each direction, and the type of the audio signal is determined. An icon
corresponding to the determination result is assigned. The created icon is displayed in the
vicinity of the position obtained from the sound source direction on the touch panel 6.
[0045]
Then, in step S6, it is determined whether an operation has been performed on the icon. If it is
determined that there is no operation on the icon, the process returns to the process of step S1
(sound collection by the microphone array 11), and the processes of steps S2 to S5 as described
above are performed.
[0046]
On the other hand, when it is determined in step S6 that there is an operation on the icon,
processing of the sound signal of the sound source corresponding to the operated icon is
performed in step S7. The voice removal device 16 removes the voice signal. In addition, you
may make it perform not only removal but the process of the suppression which reduces the
level of an audio ¦ voice signal. As described above, the audio signal of the sound source that is
detected during the imaging and considered unnecessary by the photographer is removed and
supplied to the storage device 5 together with the video signal.
[0047]
As an example of the operation to the icon, there is a touch panel 6 in which the position of the
icon corresponding to the sound source to be removed is touched with the fingertip, and the
fingertip is moved in the direction outside the display area to move the icon. Furthermore, an
operation of pinching an icon with a fingertip may be an operation of removing a sound source
corresponding to the icon. The size of the icon may be reduced in conjunction with the operation
of pinching with a fingertip, and the level of the sound source corresponding to the icon may be
suppressed in proportion to the size of the icon. Other than these, drag and drop operations used
in mobile terminals, mobile phones and the like can be adopted.
03-05-2019
14
[0048]
First Example of Icon Display A first example of the icon display will be described with reference
to FIG. FIG. 4A is an example of the subject during imaging, and FIG. 4B is a display image of the
touch panel 6 of the imaging device. When shooting a seaside subject as shown in FIG. 4A, the
sound collected by the microphone array 11 is the voice of the person (child) 31, the sound of
the waves of the sea 32, and the sound of the tree 33 caused by the wind.
[0049]
The sound sources of these sounds are separated for each direction, and their sizes are detected
to form icons corresponding to the types and sizes of the sound sources, so the display area of
the touch panel 6 is shown in FIG. 4B. An indication like that is made. That is, an icon I1 imitating
a human face is displayed in the vicinity of the person 31, and a wave-shaped icon I2 is displayed
at the position of the sea. The sound generated in the tree 33 by the wind can not specify the
type of the sound source, so the icon I3 assigned to the unspecified sound source is displayed.
[0050]
The sizes of the icons I1, I2 and I3 represent the sizes of the respective sound sources. In the
example of FIG. 4B, the magnitude relation of (sound generated by tree 33 (icon I3)> sound of
wave of sea 32 (icon I2)> voice of person 31 (icon I1)) is represented by the size of these icons.
ing. Furthermore, in order to prevent the subject from being hidden by the icon, the icon is
displayed at a position slightly shifted from the position of the sound source on the screen. Such
consideration is particularly necessary when the subject is a person.
[0051]
Second Example of Icon Display A second example of icon display will be described with
reference to FIG. FIG. 5A is an example of a subject being imaged, and FIG. 5B is a through image
displayed on the touch panel 6 of the imaging device. The image of FIG. 5A is similar to FIG. 4A.
However, as shown in FIG. 5A, the sound source is present outside the display area (the area of
03-05-2019
15
the through image) of the touch panel 6.
[0052]
That is, the voice of the other person 34 on the lower right side outside the through image is
collected toward the through image, and the music flowing from the loudspeaker 35 grounded
on the outside left side of the through image toward the through image It is collected. The other
sound sources are the same as in FIG. 4A. That is, the sound collected by the microphone array
11 is the voice of the person (child) 31, the sound of the waves of the sea 32, the sound of the
tree 33 caused by the wind, the voice of the other person 34, and the music from the
loudspeaker 35.
[0053]
As shown in FIG. 5B, in the display area of the touch panel 6, the sound sources of these sounds
are separated for each direction, and the magnitudes thereof are detected to form icons
corresponding to the types and sizes of the sound sources. Ru. For sounds generated outside the
through image, an icon is displayed near the outer frame of the display area corresponding to the
generation position. For example, eight directions (upper, lower, left, right, upper left, upper
right, lower left, lower right) are defined. In the example shown in FIG. 5, the voice of another
person is detected as the lower right position, the icon I4 of the human face is displayed, the
music from the loudspeaker on the beach is detected as the left position, and the icon I5 of the
music is displayed. Be done. The magnitudes of these icons I1 to I5 correspond to relative
loudness magnitudes.
[0054]
When displaying the sound generated at the position outside the area of the through image, as
shown in FIG. 6, the icon dedicated area 35 is arranged below the image area, and the sound
source outside the area of the through image is placed in the icon dedicated area 36 The icon
corresponding to may be displayed. That is, the icon I4 'represents the voice of the other person
34 on the right side, an arrow indicating that the sound comes from the right side is added, and
the icon I5' represents the music of the loudspeaker 35 on the left side. , An arrow is added to
indicate that the sound comes from the left side.
03-05-2019
16
[0055]
An icon I6 in the form of a microphone in FIG. 6 indicates wind noise for the microphone. The
wind noise is a sound source that does not generate a subject (person or object), and therefore no
arrow indicating the direction is added. Alternatively, as shown by icons I6a, I6b, I6c, and I6d in
FIG. 6, an icon representing wind noise may be displayed at the center position of each side of
the outer frame of the through image. Furthermore, although not illustrated, the icon itself may
be blinked or shaken to make a sound source that does not generate a subject (person, object)
such as wind noise be noticeable. As described above, a sound source that does not generate a
subject (person or object) such as wind noise is often an unnecessary sound, and an icon is
displayed so as to stand out as a target of removal.
[0056]
[Third Example of Icon Display] Instead of the icon display, a character string characterizing the
sound may be displayed. For example, when an object similar to FIG. 4A (FIG. 7A) is imaged, a
character string I7 representing the sound of a wave is displayed instead of the icon
corresponding to the wave 32 of the through image of the touch panel 6. For example, the size of
the character string 17 is proportional to the size of the sound.
[0057]
Operation on Icon The user can operate a desired icon displayed on the screen of the touch
panel 6 to perform processing such as removal of a sound corresponding to the icon. As an
example, when imaging the same subject as that in FIG. 4A (FIG. 8A), icons I1, I2, and I3 are
displayed on the screen of touch panel 6, as shown in FIG. 8B.
[0058]
As the wind noise is not a sound emitted by the subject as described above, the icon I3 is touched
with a fingertip when the user during imaging wants to remove the wind noise. Then, the
fingertip is moved to the outside of the display area of the touch panel 6 while touching the icon
I 3, and the fingertip is released from the touch panel 6. The series of drag and drop operations
03-05-2019
17
are detected by the control unit 4 and, in response to the operations, the control unit 4 generates
a control signal instructing the voice removal device 16 to mute the wind noise signal. As
described above, when the user presses the recording start button after the wind noise is
removed, the storage device 5 records the voice signal from which the wind noise is removed and
the subject image.
[0059]
The first embodiment of the present disclosure described above has the following effects. By
visualizing the status of the sound collected at the time of imaging by the type of icon, the size of
the icon, etc., the photographer can immediately confirm the status of the sound. -By operating
the icon by an intuitive operation, it is possible to record only after processing the sound linked
to the icon. Therefore, unnecessary sound can be prevented from being recorded. ・ Wind cut
noise generated when the wind hits a microphone directly is generally difficult for the
photographer to understand, but the icon can notify the shooter of the wind cut occurrence
status by an icon, and measures such as removal of the wind cut are available. You can do it
easily. By displaying the icon at a position slightly offset from the position of the sound source, it
is possible to prevent the subject from being hidden by the icon, and it is possible to capture the
subject while looking at the movement.
[0060]
Second Embodiment In the first embodiment described above, the situation of the sound
collected at the time of capturing a moving image is superimposed on the through image and
displayed as an icon. According to the second embodiment of the present disclosure, at the time
of imaging, processing (removal, suppression, etc.) on the audio signal is not performed, and the
image and the audio signal at the time of imaging are stored in the storage device and read from
the storage device. Processing the voice signal. It is to be noted that both of the first embodiment
(the operation imaging mode with sound processing) and the second embodiment (the still image
pickup mode with sound) described below are provided, and the user selects one operation mode.
Configuration is also possible.
[0061]
FIG. 9 shows the configuration of the second embodiment of the present disclosure. Although the
03-05-2019
18
configuration is the same as that of the first embodiment (see FIG. 1), the sound source removal
device 16 is not connected to the sound source separation device 15. An audio signal collected
by the microphone array 11 at the time of imaging, digitized by the digitizing circuit 13, and
separated by the sound source separation device 15 is supplied to the control unit 4. The control
unit 4 stores an audio signal in the storage device 5.
[0062]
For example, an image signal corresponding to one still image (one frame) is acquired through
the path of the optical block 1, the camera unit 2 and the video processing device 3 and stored in
the storage device 5 via the control unit 4. An audio signal after separation for a predetermined
time before and / or after the time when the still image is acquired is taken into the control unit
4. Along with the audio signal, the control unit 4 also acquires information on the direction and
magnitude associated with the audio signal. The control unit 4 links (links) the audio signal (for
example, a compression-encoded audio signal) separated in each direction for a predetermined
time, the information on the direction and size of each audio signal, and the recorded image. And
information for the above are stored in the storage unit 5.
[0063]
When a desired voice-added still image is selected and reproduced from the storage device 5, the
reproduced image is displayed on the touch panel 6. An icon corresponding to each sound source
is created by the icon creation device 17 from the audio signal read from the storage device 5,
and the icon created by the icon display device 18 is displayed.
[0064]
Then, by operating the icon displayed on the touch panel 6, processing (removal or the like) on
the audio signal is performed. Although not shown, a speaker for reproducing the audio signal
reproduced from the storage device 5 is provided, and it is possible to listen to the effect of
processing on the audio signal. Furthermore, the audio signal stored in the storage device 5 may
be updated to the processed audio signal.
[0065]
03-05-2019
19
FIG. 10A shows an example of a still image to be captured (an image similar to the image shown
in FIG. 5A described above). FIG. 10B shows a display image reproduced from the storage device
5 and displayed on the touch panel 6. The voice of the other person 34 on the lower right side
outside the through image is collected toward the through image, and the music flowing from the
loudspeaker 35 grounded on the outside left side of the through image toward the through
image is collected It is done.
[0066]
Therefore, in the example of FIG. 10A, the sound collected by the microphone array 11 is the
voice of the person (child) 31, the sound of the waves of the sea 32, the sound generated in the
tree 33 by the wind, the voice of the other person 34, the loudspeaker 35 It is music from Signals
of these separated sounds at predetermined times before and after the imaging time point, and
information of the respective directions and magnitudes are linked with the image and stored in
the storage device 5.
[0067]
An image and an audio signal associated with the image are read out of the storage device 5.
Then, an icon corresponding to each sound source is created by the icon creation device 17 from
the audio signal read from the storage device 5, and the icon created by the icon display device
18 is displayed as shown in FIG. 10B. That is, in the display area of the touch panel 6, an icon I1
imitating a human face is displayed in the vicinity of the person 31, and an icon I2 in the form of
a wave is displayed at the position of the sea. The sound generated in the tree 33 by the wind can
not specify the type of the sound source, so the icon I3 assigned to the unspecified sound source
is displayed. The sizes of the icons I1, I2 and I3 represent the sizes of the respective sound
sources.
[0068]
For sounds generated outside the through image, an icon is displayed near the outer frame of the
display area corresponding to the generation position. The voice of another person is detected as
the lower right diagonal position, the icon I4 of the human face (hidden by the hand) is displayed,
03-05-2019
20
the music from the loudspeaker on the beach is detected as the left position, and the music icon
I5 is Is displayed. For example, the user touches an icon corresponding to a sound to be removed
with a fingertip, slides the fingertip outside the display area of the touch panel 6, and then
releases the fingertip from the screen of the touch panel 6. This operation is detected by the
control unit 4 and the control unit 4 performs a process of removing the sound. Therefore, the
target voice of the removal process is not reproduced. Furthermore, it is also possible that the
control unit 4 accesses the storage device 5 and erases the operation target sound from the
storage device 5. Unlike the method of reproducing the sounds of all the sound sources and
removing the desired sound as the processing at the time of reproduction, the processing of
reproducing only the sound corresponding to the operated icon without reproducing the sounds
of all the sound sources You may do so.
[0069]
According to the second embodiment of the present disclosure described above, it is possible to
create a still image with sound, and to remove only the sound corresponding to the icon operated
at the time of reproduction, or to reproduce only the sound It becomes possible. Furthermore, it
is also possible to reproduce only the sound stored in the storage device 5 and search for a
desired image.
[0070]
Note that the present disclosure can also have the following configurations. (1) A display unit for
displaying an acquired image, An audio direction detection unit for separating an acquired audio
into audio signals for each audio signal generation source, and detecting a direction for each
audio signal generation source, The audio A figure or a character is generated corresponding to
the voice signal generation source detected by the direction detection unit, the detected direction
is converted to a position on the display area of the display unit, and the figure or the shape is
displayed near the position. An imaging control apparatus comprising: a control unit that
displays characters. (2) The imaging control device according to (1), which displays the figure or
the character corresponding to substantially the same direction as the direction in which the
audio signal generation source exists when the position of the audio signal generation source is
outside the display area . (3) When the position of the audio signal generation source is outside
the display area, either the graphic or the character is displayed in the graphic or character
display area provided in the display area (1) or (2) The imaging control apparatus as described.
(4) The imaging control device according to any one of (1), (2) and (3), wherein the figure or
character represents the strength and type of sound generated by the sound signal generation
03-05-2019
21
source to which the figure or character corresponds. (5) A voice operation unit for removing,
suppressing or emphasizing voice generated by a voice signal generation source corresponding
to the figure or character by manipulating the figure or character (1), (2), ( 3) and the imaging
control apparatus in any one of (4). (6) The display unit is configured as a touch panel, and by
operating the figure or character on the touch panel, removal, suppression or enhancement of
sound generated by an audio signal generation source corresponding to the figure or character is
performed. The imaging control apparatus according to (5), having a voice operation unit. (7) The
imaging control apparatus according to any one of (5) and (6), further including: a recording unit
configured to record the acquired image data and audio data processed by the audio operation
unit on a recording medium. (8) The acquired image is displayed on the display unit, and the
acquired audio is separated by the audio direction detection unit into an audio signal for each
audio signal generation source, and the direction for each audio signal generation source is
detected and controlled. The unit generates a figure or character corresponding to the voice
signal generation source detected by the voice direction detection unit, converts the detected
direction to a position on the display area of the display unit, and The imaging control method
which displays the said figure or character in the vicinity. (9) The computer displays the acquired
image on the display unit, and the acquired audio is separated by the audio direction detection
unit into an audio signal for each audio signal generation source, and the direction for each audio
signal generation source The control unit generates a figure or a character corresponding to the
audio signal generation source detected by the audio direction detection unit, and converts the
detected direction to a position on the display area of the display unit. And a program for
executing an imaging control method for displaying the figure or character near the position.
(10) An imaging unit for acquiring an image, a plurality of microphones for acquiring audio, a
display unit for displaying an image acquired by the imaging unit, and audio of each audio signal
generation source acquired by the microphone A voice direction detection unit that separates
into signals and detects the direction of each voice signal generation source, and generates a
figure or a character corresponding to the voice signal generation source detected by the voice
direction detection unit, and is detected Control unit that converts the display direction to a
position on the display area of the display unit and displays the graphic or character in the
vicinity of the position; voice corresponding to the graphic or character by operating the graphic
or character A voice operation unit for removing, suppressing or emphasizing the voice
generated by the signal generation source, the acquired image data, and the voice data processed
by the voice operation unit An imaging apparatus having a recording unit for recording on a
recording medium. (11) A voice direction detection unit that separates voice of a predetermined
time acquired together with a still image into voice signals for each voice signal generation
source, detects a direction of each voice signal generation source, the still image, and the voice A
recording unit for recording an audio signal for each signal generation source and information
on a direction for each audio signal generation source on a recording medium; a still image from
the recording medium; an audio signal for each audio signal generation source; A reproduction
03-05-2019
22
unit for reproducing information of a direction, a display unit for displaying the still image
reproduced from the recording medium by the reproduction unit, and audio for each of the audio
signal generation sources reproduced from the recording medium by the reproduction unit A
control unit configured to generate a figure or character in response to a signal, convert the
reproduced direction information into a position on a display area of the display unit, and display
the figure or character in the vicinity of the position Image control device. (12) A voice direction
detection unit that separates voice of a predetermined time acquired together with a still image
into voice signals for each voice signal generation source, detects a direction of each voice signal
generation source, the still image, and the voice A recording unit for recording, on a recording
medium, an audio signal for each signal generation source and position information obtained by
converting information on a direction of each audio signal generation source into a position on a
display area of the display unit; A display unit for displaying the still image reproduced from the
recording medium; a graphic or character corresponding to an audio signal for each of the audio
signal generation sources reproduced from the recording medium by the reproducing unit; And a
control unit configured to display the graphic or the character in the vicinity of the position
designated by the position information.
[0071]
Although the embodiments of the present disclosure have been specifically described above, the
present invention is not limited to the above-described embodiments, and various modifications
can be made based on the technical idea of the present disclosure. For example, the types,
shapes, and the like of the icons in the above-described embodiment are merely examples.
[0072]
DESCRIPTION OF SYMBOLS 2 ... Camera part 3 ... Video processing apparatus 4 ... Control part 5
... Storage device 6 ... Touch panel 11 ... Microphone array 14 ... Sound source direction detection
apparatus 15 ... Sound source separation Device 16 ··· Speech removal device 17 ··· Icon creation
device 18 ··· Icon display device I1 to I6 ··· Icon I7 ··· Character string
03-05-2019
23
1/--страниц
Пожаловаться на содержимое документа