close

Вход

Забыли?

вход по аккаунту

JP2010056589

код для вставкиСкачать
Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2010056589
An apparatus for performing processing that requires adjustment in accordance with the rotation
of a user's head is capable of appropriately detecting the rotation of the user's head and
performing appropriate adjustment. SOLUTION: A gyro sensor 16 is provided for an earphone
15L attached to the user's ear to detect rotation of the user's head, and an acceleration sensor 17
is provided for the earphone 15L to incline the gyro sensor 16. To detect The sound image
localization correction unit 122 corrects the detection output of the gyro sensor 16 based on the
detection output of the acceleration sensor 17. The sound image localization processing
performed in the sound image localization processing unit is adjusted by the corrected detection
output of the gyro sensor 17 so that the localization position of the sound image is always
constant. [Selected figure] Figure 1
Audio processing apparatus, sound image localization position adjustment method, video
processing apparatus and video processing method
[0001]
The present invention is an apparatus for processing audio and video that performs processing
that requires adjustment in accordance with the rotation of the head of the user (user), such as
sound image localization processing and adjustment processing of a video cutout angle, and It
relates to the method used in the device.
[0002]
Audio signals accompanying videos such as movies are recorded on the assumption that they are
10-05-2019
1
reproduced by speakers placed on both sides of the screen.
According to this, the position of the sound source in the image matches the position of the
sound image that is actually heard, and a sound field having a natural spread is established.
[0003]
However, when such audio signals are viewed using headphones or earphones, the sound image
is localized in the head, and the direction of the image and the localization position of the sound
image do not match, resulting in localization of a very unnatural sound image. It will
[0004]
In addition, the same applies to the case of listening to music without video, and unlike the case
of speaker reproduction, the music to be reproduced is heard from the inside of the head,
resulting in an unnatural reproduction sound field.
[0005]
As a means for avoiding such in-head localization of reproduced speech, a method of creating a
virtual sound image by means of head related transfer function (HRTF) is known.
[0006]
FIGS. 8 to 11 are diagrams for explaining an outline of virtual sound image localization
processing using head related transfer functions.
Here, the case where the virtual sound image localization processing is applied to a headphone
system of two channels on the left and right is shown.
[0007]
As shown in FIG. 8, the headphone system of this example includes a left channel audio input
terminal 101L and a right channel audio input terminal 101R.
[0008]
10-05-2019
2
The signal processing unit 102, the left channel D / A converter 103L, the right channel D / A
converter 103R, the left channel amplifier 104L, the right channel amplifier 104R, and the left
stage are provided downstream of the voice input terminals 101L and 101R. A headphone
speaker 105L and a right headphone speaker 105R are provided.
[0009]
Then, digital audio signals input through the audio input terminals 101L and 101R are supplied
to the signal processing apparatus 102, where signal processing for localizing the sound image
created by each audio signal at an arbitrary position (virtual sound image localization processing)
Will be applied.
[0010]
The left and right digital audio signals subjected to virtual sound image localization processing in
the signal processing device 102 are converted into analog audio signals in the D / A converters
103L and 103R.
The left and right audio signals converted into analog audio signals are amplified by the
amplifiers 104L and 104R and then supplied to the headphone speakers 105L and 105R.
As a result, audio corresponding to the left and right two-channel audio signals subjected to the
virtual sound image localization processing is emitted from the headphone speakers 105L and
105R.
[0011]
Furthermore, as will be described later, a gyro sensor 106 for detecting the rotation of the head
of the user is provided on a headband for mounting the left and right headphone speakers 105L
and 105R on the head of the user.
[0012]
The detection output from the gyro sensor 106 is supplied to the detection unit 107, where the
angular velocity is detected when the user rotates the head.
10-05-2019
3
The angular velocity from the detection unit 107 is converted to a digital signal by the A / D
conversion device 108 and then supplied to the calculation unit 109.
In the calculation unit, the correction value of the head related transfer function is calculated
according to the angular velocity at the time of rotation of the user's head, and this is supplied to
the signal processing unit 102 to correct the localization of the virtual sound image.
[0013]
As described above, by detecting the rotation of the head of the user using the gyro sensor, the
localization position of the virtual sound image can always be localized at a predetermined
position according to the direction of the head of the user. ing.
[0014]
That is, when the head of the user rotates, the localization position of the virtual sound image is
not localized in front of the user, but even if the head of the user rotates, the localization position
of the sound image remains at the original position. To be
[0015]
Then, in the signal processing apparatus 102 shown in FIG. 8, transfer functions HLL, HLR, from
the two speakers SL and SR placed in front of the listener M to both ears of the listener M as
shown in FIG. Transfer characteristics corresponding to HRR and HRL are given.
[0016]
Here, the transfer characteristic HLL is a transfer characteristic from the speaker SL to the left
ear YL of the listener M.
The transfer characteristic HLR is a transfer characteristic from the speaker SL to the right ear
YR of the listener M.
The transfer characteristic HRR is a transfer characteristic from the speaker SR to the right ear
10-05-2019
4
YR of the listener M.
The transfer characteristic HRL is a transfer characteristic from the speaker SR to the left ear YL
of the listener M.
[0017]
These transfer functions HLL, HLR, HRR, and HRL can be obtained as impulse responses on the
time axis.
By realizing this impulse response in the signal processing device 102 of FIG. 8, even when the
reproduced sound is heard with headphones, as shown in FIG. 9, the speakers SL and SR placed
at the front position of the listener M. It is possible to reproduce a sound image equivalent to that
produced by
[0018]
As described above, the process of adding the transfer functions HLL, HLR, HRR, and HRL to the
audio signal to be processed is realized by the FIR filter provided in the signal processing device
102 of the headphone system shown in FIG. Be done.
[0019]
Specifically, the signal processing device 102 shown in FIG. 8 is configured as shown in FIG.
That is, for an audio signal input through the audio input terminal 101L of the left channel, an
FIR filter 1021 for realizing the transfer function HLL and an FIR filter 1022 for realizing the
transfer function HLR are provided.
[0020]
Further, for an audio signal input through the audio input terminal 101R of the right channel, an
FIR filter 1023 for realizing the transfer function HRL and an FIR filter 1024 for realizing the
transfer function HRR are provided.
10-05-2019
5
[0021]
The output signal from the FIR filter 1021 and the output signal from the FIR filter 1023 are
subjected to addition processing by the adder 1025 and supplied to the left headphone speaker
105L.
The output signal from the FIR filter 1024 and the output signal from the FIR filter 1022 are
subjected to addition processing by the adder 1026 and supplied to the right headphone speaker
105R.
[0022]
With the signal processing apparatus 102 having such a configuration, transfer functions HLL
and HLR are added to the audio signal of the left channel, and transfer functions HRL and HRR
are added to the audio signal of the right channel. .
[0023]
Also, by using the detection output of the gyro sensor 106 mounted on the headband, even when
the user's head is rotated, the localization position of the virtual sound image is maintained at a
constant position, and the sound formed by the reproduced voice It is made possible to form a
place as a natural sound field.
[0024]
Also, here, the case where the virtual sound image localization processing is applied to the sound
signals of the left and right two channels has been described as an example.
However, the audio signal to be processed is not limited to the audio signals of the two left and
right channels.
Patent Document 1 described later describes in detail the invention of an audio reproduction
apparatus in which virtual sound image localization processing is performed on audio signals of
10-05-2019
6
multiple channels.
[0025]
In addition, said patent document 1 is as showing below.
Unexamined-Japanese-Patent No. 11-205892
[0026]
By the way, in the case of the conventional headphone system which performs the virtual sound
image localization processing described with reference to FIGS. 8 to 10, the gyro sensor 106
detects the rotation of the head of the user. Is used. And in the case of the conventional
headphone system, it is possible to provide the gyro sensor 106 on the headphone with its
detection axis in the vertical direction (gravity direction).
[0027]
That is, as shown in FIG. 11, the gyro sensor 106 can fix the left and right headphone speakers
105L, 105R at a predetermined position of the headband 110 for fixing the head on the user's
head. Thus, when the headphone system is worn on the head of the user, the detection axis of the
gyro sensor 106 can be maintained in the vertical direction.
[0028]
However, so-called insert-type or intra-conca type earphones, which are used by being inserted
into the user's ear shells, which do not have a headband, or so-called ear-hook types that use a
speaker over the user's ear shells It can not be applied to headphones as it is.
[0029]
In the case of an insert-type or intra-concha-type earphone or ear-hook-type headphones, the
10-05-2019
7
shape of the ear varies depending on the user, and the manner of attachment also varies
depending on the user.
Therefore, attach a gyro sensor to the insertion-type or intraconca-type earphone or ear-hooktype headphone so that the detection axis is in the vertical direction when these are attached to
the user's ear Is virtually impossible.
[0030]
The same is true, for example, in a system for changing an image to be displayed according to the
rotation of the user's head in a small display device used by being attached to the head of the
user called a head mounted display or the like. Also occur.
[0031]
That is, in the case of the head mounted display, when the rotation of the head of the user can
not be accurately detected, it may not be possible to display an appropriate video according to
the direction of the head of the user.
[0032]
In view of the above, the present invention enables a device that performs processing that
requires adjustment in accordance with the rotation of the user's head to appropriately detect the
rotation of the user's head and perform appropriate adjustment. The purpose is
[0033]
According to a first aspect of the present invention, there is provided a sound processing
apparatus according to the first aspect of the present invention, comprising: sound image
localization processing means for performing sound image localization processing on an audio
signal to be reproduced according to a head transfer function obtained in advance. A speaker
unit which is attached to the user's ear, receives the supply of the audio signal subjected to the
sound image localization processing by the sound image localization processing means, and
emits the sound according to the audio signal; Rotation detection means for detecting the
rotation of the head of the user on which the speaker unit is mounted; tilt detection means
provided for the speaker unit for detecting the tilt of the rotation detection means; Rotation
correction means for correcting the detection result from the rotation detection means based on
the detection result of the detection means; and the sound based on the detection result from the
rotation detection means corrected by the rotation correction means. Controls localization
processing unit, in which an adjusting means for adjusting the localization position of the sound
10-05-2019
8
image.
[0034]
According to the voice processing apparatus of the first aspect of the present invention, the
rotation detection means provided for the speaker unit worn on the user's ear detects the
rotation of the user's head, and the speaker unit The inclination of the rotation detection means
is detected by the inclination detection means provided.
[0035]
The detection output from the rotation detection means is corrected by the rotation correction
means based on the inclination of the rotation detection means from the inclination detection
means.
The sound image localization processing in the sound image localization processing means is
adjusted by the detection output from the corrected rotation detection means, and the
localization position of the sound image is adjusted.
[0036]
Thus, the rotation of the user's head can be appropriately detected, the sound image localization
process in the sound image localization processing means can be appropriately adjusted, and the
localization position of the sound image can be properly adjusted.
[0037]
According to the present invention, it is possible to appropriately detect the rotation of the user's
head and perform appropriate adjustment in a device that performs processing that requires
adjustment in accordance with the rotation of the user's head.
[0038]
Hereinafter, an embodiment of the present invention will be described with reference to the
drawings.
[0039]
10-05-2019
9
First Embodiment The present invention can be applied to a multi-channel audio processing
device in principle.
However, in the first embodiment described below, in order to simplify the description, a case
where it is applied to the left and right two-channel audio processing apparatus will be described
as an example.
[0040]
FIG. 1 is a block diagram for explaining a configuration example of the earphone system 1
according to the first embodiment.
The earphone system 1 shown in FIG. 1 roughly includes an audio signal reproduction system
and a detection and correction system for the rotation of the head of the user.
[0041]
The audio signal reproduction system includes a music / audio reproduction device 11, a sound
image localization processing unit 121 of the signal processing processor 12, D / A (Digital /
Analog) converters 13L and 13R, amplifiers 14L and 14R, and earphones 15L and 15R. It is a
part.
[0042]
The part consisting of the D / A (Digital / Analog) converter 13L, the amplifier 14L and the
earphone 15L is for the left channel, and the part consisting of the D / A (Digital / Analog)
converter 13R, the amplifier 14R and the earphone 15R is It is for the right channel.
[0043]
The detection and correction system of the rotation of the head of the user is a portion including
the gyro sensor 16, the acceleration sensor 17, the A / D converter 18, and the sound image
localization correction processing unit 122 of the signal processor 12.
10-05-2019
10
[0044]
Here, the music / voice reproducing apparatus 11 is, for example, an IC recorder using a
semiconductor as a recording medium, a portable telephone terminal provided with a music
reproducing function, a compact light called CD (Compact Disc), MD (registered trademark) or
the like. It is various reproduction ¦ regeneration apparatuses, such as a reproduction ¦
regeneration apparatus of a magnetic disc.
[0045]
The earphones 15L and 15R are of an insertion type, an intraconca type, or a hooked type.
That is, the earphones 15L and 15R have various position states when worn on the ear
depending on the shape of the user's ear and how to wear it.
[0046]
The gyro sensor 16 and the acceleration sensor 17 are provided to one of the earphones 15L and
15R, and to the earphone 15L for the left channel, which will be described later in the first
embodiment.
[0047]
Then, in the earphone system 1 shown in FIG. 1, the reproduced digital sound signal from the
music and sound reproduction device 11 is supplied to the sound image localization processing
unit 121 of the signal processing processor 12.
[0048]
The sound image localization processing unit 12 has, for example, the configuration described
using FIG.
That is, as described with reference to FIG. 10, the sound image localization processing unit 121
includes four FIR filters 1021, 1022, 1023, and 1024 in which transfer functions HLL, HLR, HRL,
and HRR are set, and two adder circuits 1025. , 1026 and the like.
10-05-2019
11
[0049]
Each of the FIR filters 1021, 1022, 1023, and 1024 of the sound image localization processing
unit 121 can correct the transfer function according to the correction information from the
sound image localization correction processing unit 122 described below. is there.
[0050]
For the sound image localization correction processing unit 122 of the first earphone system 1,
as shown in FIG. 1, the detection output from the gyro sensor 16 and the detection output from
the acceleration sensor 17 are A / D converters. The digital signal is converted into a digital
signal by 18 and supplied.
[0051]
As described above, the gyro sensor 16 and the acceleration sensor 17 are provided for the left
channel speaker unit 15L.
[0052]
The gyro sensor 16 detects the horizontal rotation of the head of the user wearing the speaker
unit 15L with respect to the ear, and has, for example, one axis.
The acceleration sensor 17 has three axes, and detects the inclination of the gyro sensor 16 by
detecting the acceleration in the directions of three axes orthogonal to each other.
[0053]
Then, in order to accurately detect the horizontal rotation of the user's head, when the speaker
unit 15L is attached to the user's ear, the detection axis of the gyro sensor 16 is in the vertical
direction. Need to be provided for the speaker unit 15L.
[0054]
However, as described above, the earphones 15L and 15R are of the insertion type, the intra
concha type, the ear hook type, and the like.
10-05-2019
12
Therefore, when the earphone 15L is attached to the user's ear, the gyro sensor 16 provided in
the earphone 15L has its detection axis in the vertical direction (in other words, the detection
axis is directed to the floor surface) Often can not be done).
[0055]
Therefore, the sound image localization correction processing unit 122 detects the tilt of the gyro
sensor 16 using the detection output of the three-axis acceleration sensor 17 similarly provided
in the earphone 15L.
Then, the sound image localization correction processing unit 122 corrects the detection output
of the gyro sensor 16 based on the detection output of the acceleration sensor 17 to accurately
detect the horizontal rotation (direction and amount) of the user's head accurately. Do.
[0056]
The sound image localization correction processing unit 122 transmits the FIR filters of the
sound image localization processing unit 121 so that the sound image localization processing can
be appropriately performed according to the rotation of the head of the user accurately obtained
in this manner. Correct the function.
[0057]
As a result, even if the user wearing earphones 15L and 15R on his ear rotates his head in the
horizontal direction and changes the direction of his head, the localization position of the sound
image does not change with the movement of the user's head. The sound image can be localized
at the original position.
[0058]
That is, when listening to the emitted sound from the speaker installed in the room, the position
of the speaker does not change even if the user changes the head direction, so the emitted sound
is from the direction of the installed speaker hear.
[0059]
10-05-2019
13
However, in the case of the earphone system, when the virtual sound image localization process
is performed so that the sound image is localized in the front direction of the user, the sound
image is always localized in the forward direction of the user along with the change of the head
direction. It will
[0060]
That is, in the case of the earphone system accompanied with virtual sound image localization
processing, when the user wearing the earphone changes the direction of the head, the
localization position of the sound image also moves accordingly, resulting in an unnatural
reproduced sound field.
[0061]
Therefore, as described above, by appropriately correcting the virtual sound image localization
processing according to the rotation of the user's head in the horizontal plane by the function of
the sound image localization correction processing unit 122 or the like, the localization position
of the sound image is always constant. To create a natural playing field.
[0062]
Next, the process performed by the sound image localization correction processing unit 122 will
be specifically described.
FIG. 2 is a diagram for explaining the relationship between the detection axis of the gyro sensor
16 and the detection axis of the acceleration sensor 17 when the earphones 15L and 15R are
attached to the user's ear.
Among them, FIG. 2A is a view of the user with the earphones 15L and 15R attached from behind
and FIG. 2B is a view of the user with the earphones 15L attached and viewed from the left side.
[0063]
In FIG. 2, an axis Xa, an axis Ya, and an axis Za are three detection axes of the acceleration sensor
17 which are orthogonal to each other.
10-05-2019
14
The vertical axis Va is the original vertical direction (gravity direction) and indicates the direction
perpendicular to the floor surface.
[0064]
The acceleration sensor 17 is provided so as to have a predetermined positional relationship with
the gyro sensor 16 so that the inclination of the gyro sensor 16 can be detected.
In the earphone system 1 of the first embodiment, it is assumed that the acceleration sensor 16
is provided such that the Za axis among the three axes coincides with the detection axis of the
gyro sensor 16.
[0065]
As described above, the earphones 15L, 15R of the earphone system 1 are of the insertion type,
the intra concha type, or the ear hook type.
For this reason, as shown in FIG. 2A, each of the earphones 15L and 15R is independently
attached to each of the left and right ears of the user.
[0066]
Then, as shown in FIG. 2A showing the case where the user is viewed from behind, when the
earphone 15L is attached to the user's ear, the detection axis of the gyro sensor 16 coincident
with the Za axis of the acceleration sensor 17 is Consider the case where the vertical axis Va does
not coincide with the vertical direction.
[0067]
In this case, the amount of deviation of the detection axis of the gyro sensor 16 with respect to
the vertical direction is φ degrees as shown in FIG. 2A.
10-05-2019
15
That is, in the plane formed by the Ya axis and the Za axis which are detection axes of the
acceleration sensor 17, the deviation amount of the detection axis of the gyro sensor 16 with
respect to the vertical direction is φ degree.
[0068]
In this case, when the user is viewed from the left side, as shown in FIG. 2B, the detection axis of
the gyro sensor 16 coincident with the Za axis of the acceleration sensor 17 deviates from the
vertical direction indicated by the vertical axis Va as shown in FIG. It becomes theta degree as
shown in.
[0069]
The relationship between the detection axis of the gyro sensor 16 shown in FIGS. 2A and 2B, the
detection axes of the three axes of the acceleration sensor 17, and the vertical direction is
summarized.
FIG. 3 is a diagram for explaining the deviation between the detection axis of the gyro sensor 16
and the vertical direction in the coordinate system formed by the three detection axes Xa, Ya, Za
of the acceleration sensor 17. As shown in FIG.
[0070]
In FIG. 3, an arrow SXa on the Xa axis is a detection output of the acceleration sensor 17 in the
Xa axis direction, and an arrow SYa on the Ya axis is a detection output of the acceleration sensor
17 in the Ya axis direction. An arrow SZa is a detection output of the acceleration sensor 17 in
the Za-axis direction.
[0071]
Further, in FIG. 3, the vertical axis Va indicated by the solid line arrow is the actual vertical
direction of the coordinate system of the three axes shown in FIG. 3.
10-05-2019
16
Then, as described above, the acceleration sensor 17 is provided such that the Za axis, which is
one of the detection axes, coincides with the detection axis of the gyro sensor 16.
[0072]
Therefore, the vertical direction in the Ya-Za plane formed by the Ya axis and the Za axis of the
acceleration sensor 16 is the direction indicated by the dotted arrow VY in FIG.
Therefore, it is understood that the deviation between the detection axis of the gyro sensor 16
(corresponding to the Za axis) in the Ya-Za plane and the vertical direction VY is φ degree which
is an angle formed by the Za axis and the vertical direction VY. .
The state shown in the Ya-Za plane is the state shown in FIG. 2A.
[0073]
Further, the vertical direction in the Xa-Za plane formed by the Xa axis and the Za axis of the
acceleration sensor 16 is the direction indicated by the dotted arrow VX in FIG.
Therefore, it is understood that the deviation between the detection axis of the gyro sensor 16
(corresponding to the Za axis) in the Xa-Za plane and the vertical direction VX is θ degree which
is an angle formed by the Za axis and the vertical direction VX. .
The state shown in the Xa-Za plane is the state shown in FIG. 2B.
[0074]
Then, as can be seen from the state shown in FIG. 3, the amount of deviation of the detection axis
of the gyro sensor 16 with respect to the vertical direction in the Xa-Za plane is (cos θ).
10-05-2019
17
Similarly, the amount of deviation of the detection axis of the gyro sensor 16 with respect to the
vertical direction in the Ya-Za plane is (cos φ).
[0075]
FIG. 4 is a diagram showing an equation for explaining the correction processing performed in
the sound image localization correction processing unit 122. As shown in FIG.
[0076]
Here, the output of the gyro sensor 16 in the ideal state, that is, the detection output of the gyro
sensor 16 when the detection axis of the gyro sensor 16 coincides with the actual vertical
direction, is Si .
[0077]
Further, as shown in FIGS. 2 and 3, the output of the actual gyro sensor 16, that is, the detection
axis of the gyro sensor 16 is φ degree in the Ya-Za plane and θ in the Xa-Za plane with respect
to the vertical direction. The detection output of the gyro sensor 16 in the case of a degree of
deviation is referred to as Sr .
[0078]
In this case, the actual detection output Sr is, as shown in equation (1) in FIG. 4, the detection
output Si in the ideal state, the shift amount (cos θ) in the Xa-Za plane, and the shift amount in
the Ya-Za plane ( It can obtain ¦ require by multiplication with cos (phi).
[0079]
Then, the estimated output value of the gyro sensor 16 in the ideal state is set to
Sii
.
The estimated output value "Sii" and the output value "Si" of the gyro sensor 16 in the ideal state
are as close as possible.
[0080]
10-05-2019
18
Therefore, the estimated output value of the gyro sensor 16 in the ideal state "SII" can be
obtained by equation (2) in FIG.
That is, the estimated output value SII is the product of the actual output value Sr of the
gyro sensor 16 and the deviation amount (cos θ) in the Xa-Za plane and the deviation amount
(cos φ) in the Ya-Za plane. It is obtained by division (division).
[0081]
The sound image localization correction processing unit 122 receives the detection output from
the gyro sensor 16 and the detection output from the acceleration sensor 17.
Then, as described with reference to FIGS. 2 and 3, the sound image localization correction
processing unit 122 determines the amount of deviation of the detection axis of the gyro sensor
16 with respect to the vertical direction from the detection output of the three axes of the
acceleration sensor 17. The detected output of the gyro sensor 16 is corrected according to the
equation (2) in FIG.
[0082]
Then, the sound image localization correction processing unit 122 corrects the transfer function
of the angle FIR filter of the sound image localization processing unit 121 based on the corrected
detection output of the gyro sensor 16, and the virtual sound image is corrected according to the
rotation of the user's head. Correct the localization position of to be an appropriate position.
[0083]
As described above, the acceleration sensor 17 has three axes, and the values of tanθ and tanφ
can be determined from the output values of the two axes forming the corresponding surface,
and arctangent (arctan) can be determined from these values. The values of θ and φ can be
determined by determining
[0084]
In other words, in the state shown in FIG. 3, θ can be obtained by arctan (SZa / SXa).
10-05-2019
19
Similarly, φ can be determined by arctan (SZa / SYa).
[0085]
Thus, cos θ and cos φ can be obtained based on the detection output of the acceleration sensor
23.
Then, using these, the detection output of the gyro sensor 22 can be corrected according to the
equation (2) in FIG. 4.
[0086]
As described above, even when the detection axis of the gyro sensor 16 is not in the vertical
direction when the earphone 15L is attached to the user's ear, the acceleration provided with a
fixed positional relationship with the gyro sensor 16 The detection output of the sensor 17 can
be appropriately corrected.
[0087]
Thereby, the sound image localization processing in the sound image localization processing unit
121 is appropriately corrected according to the horizontal rotation of the head of the user, and
the localization position of the sound image is always maintained at a constant position, and a
natural reproduction sound field is obtained. It can be formed.
[0088]
In the earphone system 1 according to the first embodiment, when a predetermined operation
button switch of the earphone system 1 is operated, sound image localization processing is also
performed in consideration of the rotation of the head of the user in the horizontal direction. It
will be.
In this case, the position of the head of the user at the time when the predetermined operation
button switch is operated is the position facing the front (reference position).
10-05-2019
20
[0089]
In addition to this, for example, when the music playback button is operated, the position of the
user's head at that time is also referred to as a position facing the front (reference position), and
the user's head is also rotated. It is also possible to start the sound image localization process
considered.
[0090]
Also, for example, when the user swings the head largely to the left and right, and then detects
that the movement of the user's head is settled, the position where the position of the user's head
at that time is facing forward (reference As the position), it is also possible to start sound image
localization processing in consideration of the rotation of the head of the user.
[0091]
Besides, various triggers that can be detected by the earphone system 1 can be used to start
sound image localization processing in consideration of the rotation of the head of the user.
[0092]
Further, as understood from the above description, for example, even when the head of the user
wearing the earphones 15L and 15R is inclined, the deviation between the detection axis of the
gyro sensor 22 and the vertical direction can be The detection output of the sensor 23 can be
used for detection.
[0093]
Therefore, even when the head of the user is in a tilted state, the detection output of the gyro
sensor 16 can be corrected to an appropriate value based on the detection output of the
acceleration sensor 17.
[0094]
[Modification of First Embodiment] In the above-described earphone system 1 according to the
first embodiment, the acceleration sensor 17 has three axes. However, the present invention is
not limited to this.
10-05-2019
21
The acceleration sensor 17 may be one-axis or two-axis.
[0095]
For example, the detection axis of the one-axis acceleration sensor is originally in the vertical
direction.
Then, the deviation of the detection axis of the gyro sensor from the vertical direction is detected
according to how much the detection value of the acceleration sensor of one axis in this case
differs from the original value (9.8 m / s <2>) be able to.
[0096]
Moreover, it can think similarly also when using a 2-axis acceleration sensor.
That is, even in the case of a two-axis acceleration sensor, its detection output is perpendicular to
the detection axis of the gyro sensor according to the difference from the detection output when
the acceleration sensor is provided horizontally to the floor surface. Misalignment with the
direction can be detected.
[0097]
Also, the detection output of the acceleration sensor and the amount of deviation of the detection
axis of the gyro sensor are measured beforehand by a large number of users using an earphone
system on which the gyro sensor and the one-axis or two-axis acceleration sensor are mounted.
Create a table in which measurement values are associated.
[0098]
Then, with reference to the table from the detection output of the acceleration sensor, the
deviation amount of the detection axis of the gyro sensor in the vertical direction may be
specified, and the detection output of the gyro sensor may be corrected based on this.
10-05-2019
22
[0099]
In this case, for example, a memory in which the detection output of the acceleration sensor and
the deviation amount of the detection axis of the gyro sensor with respect to the vertical
direction are associated with the memory in the sound image localization correction processing
unit 122 or the externally referable memory. Need to be created.
[0100]
Moreover, although the gyro sensor 16 mentioned above was demonstrated as a thing of 1 axis ¦
shaft, it does not restrict to this.
It is also possible to use a multi-axis gyro sensor with two or more axes.
In this case, for example, it is also possible to detect the orientation of the head of the user in the
vertical direction (vertical direction) and correct the localization of the sound image in the
vertical direction.
[0101]
Further, as described above, the present invention is suitably applied to an insertion type,
intraconca type, ear-hook type earphone system and headphone system.
However, the present invention can be applied to cases where conventional headphones having a
headband are used, which is not applicable only to these.
[0102]
In the first embodiment, as apparent from the above description, the sound image localization
processing unit 121 realizes a function as a sound image localization processing unit, and the
earphone 15L realizes a function as a speaker unit. ing.
10-05-2019
23
Further, the gyro sensor 16 realizes a function as a rotation detection means, the acceleration
sensor 17 realizes a function as a tilt detection means, and the sound image localization
correction processing unit 122 has a function as a rotation correction means and an adjustment
means. The function is realized.
[0103]
The earphone system of the first embodiment described with reference to FIGS. 1 to 4 is one to
which the sound image localization position adjustment method of the present invention is
applied.
(1) The rotation of the head of the user wearing the earphone 15L is detected through the gyro
sensor 16 provided in the earphone 15L. (2) The tilt of the gyro sensor 16 is detected through
the acceleration sensor 17 provided in the earphone 15L. (3) Based on the inclination of the gyro
sensor 16 detected by the acceleration sensor 17, the detection result on the rotation of the head
of the user detected by the gyro sensor 16 is corrected. (4) The corrected gyro sensor 16 The
sound image localization process applied to the sound signal to be reproduced is adjusted based
on the detection result of the rotation of the head of the user detected by the user, so that the
sound image localization position is adjusted. What performs the step of (4) is the sound image
localization position adjustment method of the present invention.
[0104]
Second Embodiment Next, the case where the present invention is applied to a video processing
apparatus using a small display device, which is mounted on a user's head, which is called a socalled head mounted display, is used.
[0105]
FIG. 5 is a view for explaining the appearance of the head mounted display unit 2 used in the
second embodiment.
FIG. 6 is a block diagram for explaining a configuration example of a video processing apparatus
including the head mounted display 2 according to the second embodiment.
10-05-2019
24
[0106]
As shown in FIG. 5, the head mounted display unit 2 is used by being mounted on the head of the
user, and a small display is positioned several centimeters away from the user's eyes. It is
[0107]
In such a head mounted display, it is possible to form and display an image to be displayed on a
display screen of the display located in front of the user's eyes so as to be at a position away
from the user to some extent.
[0108]
In the video reproduction apparatus 3 constituting the video processing apparatus according to
this embodiment using the head mounted display 2, as will be described later, for example,
moving image data photographed in an angle range larger than the human viewing angle is
stored in the hard disk It is held.
Specifically, moving image data captured in a range of 360 degrees in the horizontal direction is
stored and held in the hard disk.
Then, it is possible to detect horizontal rotation of the head of the user with the head mounted
display unit 2 attached thereto, and to display an image in a direction according to the direction
of the head of the user.
[0109]
Therefore, as shown in FIG. 6, the head mounted display unit 2 includes a display unit 21
configured of an LCD (Liquid Crystal Display) or the like, and a gyro sensor 22 for detecting the
rotation of the head of the user. An acceleration sensor 23 is provided.
[0110]
Further, the video reproduction device 3 for supplying a video signal to the head mounted
display 2 is configured as, for example, a hard disk recorder, a game machine, or various other
video reproduction devices.
10-05-2019
25
[0111]
Then, as shown in FIG. 6, the video reproduction device 3 of the video processing device
according to the second embodiment includes a video reproduction unit 33 provided with a hard
disk drive (hereinafter abbreviated as HDD), and a video processing unit. It has 34 and.
[0112]
Furthermore, the video reproduction device 3 includes an A / D converter 33 for receiving a
detection output from a sensor of the head mounted display unit 2, and a user direction detection
unit 34 for detecting the direction of the head of the user. It is
[0113]
Then, generally, the video processing reproduction apparatus 3 accepts the selection of the video
content to be reproduced from the user, and upon receiving the instruction to reproduce the
video, starts the reproduction processing of the video content.
[0114]
In this case, the video reproduction unit 31 reads out the selected video content (video data)
recorded on the hard disk and supplies this to the video processing unit 32.
The video processing unit 32 performs various processes such as compression and
decompression of the video content supplied thereto and conversion to an analog signal to form
a video signal to be supplied to the display unit 21 of the head mounted display 2. And supplies
this to the display unit 21.
As a result, a video according to the target video content is displayed on the display screen of the
display unit 22 of the head mounted display unit 2.
[0115]
The head mount display unit 2 is usually held on the head by a headband, and in the case of a
glasses type, the user's head is hung on a user's ear by a so-called temple Held in the department.
10-05-2019
26
[0116]
However, depending on how the head mount display unit 2 is attached to the head band, even
when the head mount display unit 2 is attached to the head of the user, the detection axis of the
gyro sensor 22 provided on the head mount display unit 2 may not be vertical. It is believed that
there is.
[0117]
Further, in the case of the head mount display unit 2 of the glasses type, it is considered that the
detection axis of the gyro sensor 22 provided thereon may not be in the vertical direction
depending on how the user puts it.
[0118]
Therefore, as shown in FIG. 6, a gyro sensor 22 and an acceleration sensor 23 are provided in the
head mount display 2 used in the video processing apparatus according to the second
embodiment.
[0119]
Similar to the gyro sensor 16 of the earphone system 1 of the first embodiment described above,
the gyro sensor 22 is a single-axis sensor for detecting the rotation of the head of the user.
[0120]
Similar to the acceleration sensor 17 of the earphone system 1 according to the first embodiment
described above, the acceleration sensor 23 has three axes, and in order to detect the inclination
of the gyro sensor 22, the acceleration sensor 23 has a fixed position with the gyro sensor 22. It
is arranged to maintain the relationship.
[0121]
Also in the second embodiment, one detection axis (for example, Za axis) of the three-axis
acceleration sensor 23 provided in the head mount display unit 2 is arranged to coincide with
the detection axis of the gyro sensor 22. There is.
[0122]
The detection output from the gyro sensor 22 and the detection output from the acceleration
sensor 23 provided in the head mount display unit 2 are supplied to the user direction detection
10-05-2019
27
unit 34 through the A / D converter 33 of the music reproduction device 3. .
[0123]
The A / D converter 33 converts the detection output from the gyro sensor 22 and the detection
output from the acceleration sensor 23 into digital signals and supplies them to the user
direction detection unit 34.
[0124]
The user direction detection unit 34 detects the detection output of the gyro sensor 22 in the
same manner as the sound image localization correction processing unit 122 in the earphone
system 1 according to the first embodiment described with reference to FIGS. It corrects based
on the detection output from.
[0125]
Specifically, as described with reference to FIG. 3, first, from the detection output of the three
axes of the acceleration sensor 23, the amount of deviation (cos θ) of the detection axis of the
gyro sensor 22 in the Xa-Za plane with respect to the vertical direction is determined. .
Then, the deviation amount (cos φ) of the detection axis of the gyro sensor 22 in the Ya-Za plane
with respect to the vertical direction is determined.
[0126]
Then, as described with reference to FIG. 4, according to equation (2) in FIG. 4, the detection
output of the gyro sensor 22 and the shift amount (cos θ, cos φ) of the detection axis of the
gyro sensor 22 with respect to the vertical direction are used. Then, the detection output of the
gyro sensor 22 is corrected.
Thereby, the designated output value Sii of the ideal state of the gyro sensor 22 is obtained,
and the direction of the head of the user is specified accordingly.
10-05-2019
28
[0127]
Then, the user direction detection unit 34 supplies the video reproduction unit 31 with
information indicating the detected head direction of the user.
As described above, moving image data captured in a range of 360 degrees in the horizontal
direction is recorded in the HDD of the video reproduction unit 31.
[0128]
For this reason, the video reproduction unit 31 reads moving image data of a section
corresponding to the direction of the user from the user direction detection unit 34, and
reproduces the data.
[0129]
FIG. 7 is a diagram for describing a reading portion of video data in a range of 360 degrees
which the video reproduction unit 31 changes in accordance with the orientation of the head of
the user.
In FIG. 7, a range surrounded by a dotted line indicated by a letter A (hereinafter referred to as a
display range A).
Is the display range of the video displayed when the head of the user faces the front.
[0130]
Then, for example, when it is detected that the user's head is turned to the front by a certain
angle in the left direction, a range surrounded by a dotted line indicated by a letter B in FIG.
Hereinafter, the display range B will be referred to.
Is read out and reproduced.
10-05-2019
29
[0131]
Similarly, when it is detected that the user's head is turned in the front direction, for example,
when it is detected that it has been rotated by an angle to the right, the range surrounded by the
dotted line indicated by the letter C in FIG. (Hereinafter, the display range C is referred to.
Is read out and reproduced.
[0132]
As described above, when the user wearing the head mounted display unit 2 faces the front, the
video data of the display range A in FIG. 7 is read and reproduced.
When the head of the user rotates to the left by an angle from the front, the video data of the
display range B in FIG. 7 is read and reproduced.
Similarly, when the head of the user rotates to the right by an angle from the front, the video
data of the display range C in FIG. 7 is read and reproduced.
[0133]
Then, in FIG. 7, when the video data in the display range B is reproduced, when the head of the
user is further rotated in the left direction, the video data in the display area on the left side is
read and reproduced. Ru.
[0134]
Similarly, in FIG. 7, when the video data in the display range C is being reproduced, if the head of
the user is further rotated in the right direction, the video data in the display area on the right
side is read and reproduced. Be done.
[0135]
10-05-2019
30
As described above, according to the horizontal rotation of the head of the user wearing the head
mounted display unit 2, the orientation of the head of the user from the video data captured in
the range of 360 degrees and stored in the HDD. The video data of the display range according to
is cut out and reproduced.
[0136]
Then, the rotation of the head of the user can be obtained by correcting the detection output of
the gyro sensor 22 by the detection output of the acceleration sensor 23, so that the direction of
the head of the user can be detected accurately.
Thus, it is possible to cut out video data of an appropriate display range and reproduce it
according to the orientation of the head of the user on which the head mounted display unit 2 is
mounted.
[0137]
In the video processing apparatus according to the second embodiment, when a predetermined
operation button switch of the video processing apparatus is operated, video display processing
is also performed in consideration of the rotation of the head of the user. Ru.
In this case, the position of the head of the user at the time when the predetermined operation
button switch is operated is the position facing the front (reference position).
[0138]
In addition to this, for example, when the video playback button is operated, the position of the
user's head at that time is also referred to as the position facing the front (reference position),
and the user's head is also rotated. It is also possible to start the video reproduction process
considered.
[0139]
Also, for example, when the user swings the head largely to the left and right, and then detects
that the movement of the user's head is settled, the position where the position of the user's head
10-05-2019
31
at that time is facing forward (reference As the position), it is also possible to start the video
display processing in consideration of the rotation of the head of the user.
[0140]
In addition, various triggers that can be detected by the video display device can be used to start
video display processing in consideration of the rotation of the head of the user.
[0141]
[Modification of Second Embodiment] In the head mount display unit 2 of the first embodiment
described above, although the acceleration sensor 23 has three axes, the present invention is not
limited to this. .
The acceleration sensor 23 may have one axis or two axes.
[0142]
For example, the detection axis of the one-axis acceleration sensor is originally in the vertical
direction.
Then, the deviation of the detection axis of the gyro sensor from the vertical direction is detected
according to how much the detection value of the acceleration sensor of one axis in this case
differs from the original value (9.8 m / s <2>) be able to.
[0143]
Moreover, it can think similarly also when using a 2-axis acceleration sensor.
That is, even in the case of a two-axis acceleration sensor, its detection output is perpendicular to
the detection axis of the gyro sensor according to the difference from the detection output when
the acceleration sensor is provided horizontally to the floor surface. Misalignment with the
10-05-2019
32
direction can be detected.
[0144]
Also, the detection output of the acceleration sensor and the amount of deviation of the detection
axis of the gyro sensor are measured beforehand by a large number of users using an earphone
system on which the gyro sensor and the one-axis or two-axis acceleration sensor are mounted.
Create a table in which measurement values are associated.
[0145]
Then, with reference to the table from the detection output of the acceleration sensor, the
deviation amount of the detection axis of the gyro sensor in the vertical direction may be
specified, and the detection output of the gyro sensor may be corrected based on this.
[0146]
In this case, for example, a table in which the detection output of the acceleration sensor and the
shift amount of the detection axis of the gyro sensor with respect to the vertical direction are
associated with the memory in the user direction detection unit 34 or the external referenceable
memory It is necessary to create it.
[0147]
Moreover, although the gyro sensor 22 mentioned above was demonstrated as a thing of 1 axis ¦
shaft, it does not restrict to this.
It is also possible to detect the orientation of the head of the user in the vertical direction
(vertical direction) using a multi-axis gyro sensor of two or more axes, and correct the
localization of the sound image in the vertical direction.
[0148]
In the second embodiment, as is apparent from the above description, the head mount display
unit 2 realizes the function of the display means, and the gyro sensor 22 realizes the function as
the rotation detection means. The acceleration sensor 23 realizes a function as tilt detection
10-05-2019
33
means.
Further, the user orientation detection unit 34 realizes a function as a rotation correction unit,
and the video reproduction unit 31 realizes a function as a video processing unit.
[0149]
The video processing apparatus according to the second embodiment described mainly with
reference to FIGS. 5 to 7 is one to which the video processing method according to the present
invention is applied.
That is, (A) the rotation of the head of the user with the head mount display unit 2 attached is
detected through the gyro sensor 22 provided in the head mount display unit 2; (B) the
acceleration sensor provided in the head mount display unit 2 The inclination of the gyro sensor
22 is detected through (23), and the detection result of the rotation of the head of the user
detected by the gyro sensor 22 is corrected based on the inclination of the gyro sensor 22
detected by the acceleration sensor 23. (D) Based on the corrected detection result of the rotation
of the head of the user detected by the gyro sensor 22, the video reproduction unit 31 records
the video data of the range corresponding to the rotation of the head of the user on the hard disk
For example, it cuts out from the image data for 360 degrees in the horizontal direction and It is
the video processing method according to the present invention that performs the steps (A) to (D)
to supply to the lay unit 2.
[0150]
<Others> Further, as described above, in the first embodiment, the earphone system 1 to which
the voice processing device of the present invention is applied, and in the second embodiment,
the head mounted display unit 2 The video processing apparatus to which the video processing
apparatus of the present invention is applied has been described.
[0151]
However, it is not limited to this.
10-05-2019
34
The present invention can be applied to an audio / video processing apparatus provided with an
audio reproduction system and a video reproduction system.
In this case, the gyro sensor and the acceleration sensor are provided with the gyro sensor and
the acceleration sensor in either one of the earphone and the head mounted display, and the
detection output of the gyro sensor is corrected based on the detection output of these sensors.
[0152]
Then, using the corrected detection output from the gyro sensor, the sound image localization
processing performed in the sound image localization processing unit is controlled, and the
display range (readout range) of the video data in the video reproduction device is controlled.
[0153]
As a result, if one gyro sensor and one acceleration sensor are provided, it is possible to
appropriately perform both the virtual sound image localization processing of sound and the
control processing of the video clipping range.
[0154]
BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram for demonstrating the structural
example of the earphone system 1 of 1st Embodiment to which one Embodiment of the audio ¦
voice processing apparatus of this invention was applied.
FIG. 17 is a diagram for describing the relationship between the detection axis of the gyro sensor
16 and the detection axis of the acceleration sensor 17 when the earphones 15L and 15R are
attached to the user's ear.
FIG. 7 is a diagram for describing the deviation between the detection axis of the gyro sensor 16
and the vertical direction in the coordinate system formed by the three detection axes Xa, Ya, Za
of the acceleration sensor 17.
It is a figure which shows the formula for demonstrating the correction process performed in the
sound image localization correction process part 122. FIG.
10-05-2019
35
It is a figure for demonstrating the external appearance of the head mounted display part 2 of
the video processing apparatus used by this 2nd Embodiment to which one embodiment of the
video processing apparatus of this invention was applied.
It is a block diagram for demonstrating the structural example of the video processing apparatus
containing the head mounted display 2 of 2nd Embodiment.
It is a figure for demonstrating the read-out part of the video data of the range of 360 degree
which the image ¦ video reproduction ¦ regeneration part 31 changes according to the direction
of a user's head.
It is a figure for demonstrating the structural example of the headphone system where virtual
sound image localization process is used.
It is a figure for demonstrating the concept of virtual sound image localization processing of 2
channels.
It is a figure for demonstrating the structural example of the signal processing part 102 shown in
FIG.
It is a figure for demonstrating the case where the conventional headphone system provided with
the gyro sensor 10 is mounted ¦ worn with a user's head.
Explanation of sign
[0155]
DESCRIPTION OF SYMBOLS 1 ... earphone system, 11 ... music and sound reproduction
apparatus, 12 ... signal processing processor, 121 ... sound image localization processing unit,
122 ... sound image localization correction unit, 13L, 13R ... D / A converter, 14L, 14R ...
amplifier, 15L , 15R: earphone, 16: gyro sensor, 17: acceleration sensor, 18: A / D converter 18,
2: head mounted display unit, 21: display unit, 22: gyro sensor, 23: acceleration sensor, 3: image
10-05-2019
36
reproduction Device 31 Video reproduction unit 32 Video processing unit 33 A / D converter 34
User direction detection unit
10-05-2019
37
1/--страниц
Пожаловаться на содержимое документа