JPH0993700

Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JPH0993700
[0001]
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a
video and audio reproduction apparatus capable of viewing video and audio by being worn on
the head.
[0002]
2. Description of the Related Art Conventionally, reproduction of a sense of reality in a video and
audio reproduction system has mainly been the enlargement of a system by large screen video
and multi-speaker reproduction. However, as shown in FIG. The video / audio reproduction
apparatus 100 has been used in which personalization and miniaturization are achieved by the
head-mounted type video display unit 101 and the audio reproduction unit 102 added thereto.
[0003]
As shown in FIG. 23, the video display unit 101 is driven by the drive circuit 104 supplied with a
video input signal from the video input terminal 103 to display a video.
[0004]
Similarly, as shown in the figure, the audio reproduction unit 102 amplifies the audio input signal
input from the audio input terminal 105 by the amplifiers 106L and 106R, and supplies it to the
left-ear sound producing body 107L and the right-ear sound producing body 107R. , Giving a
voice to the wearer.
10-05-2019
1
[0005]
Here, since the sound reproducing unit 102 localizes the sound image in the head of the wearer,
the sound reproducing unit 102 is very unnatural in the sense of reproducing the sense of reality
without being matched with the image displayed by the image display unit 101.
[0006]
Therefore, the video and audio are updated in real time according to the movement of the body,
and further, in the audio, an attempt is made to reproduce a near-real viewing environment by
convoluting, for example, with a digital signal processing unit. ing.
[0007]
By the way, in the above-mentioned attempts, a great number of signal processing is required
especially in speech processing, and it can not be made large enough to be worn on the head by
itself, and movement The detection means also had to be expensive and was not practical.
[0008]
Also, even if it is possible to realize the sound field feeling close to reality in the sound by
realizing the localization outside the head of the sound image, the video system moves in the
viewing environment following the movement of the head as in the conventional case. A large
deviation occurs between the sound image position and the video position, resulting in unnatural
video and audio reproduction.
[0009]
Further, even when the video contents are updated according to the movement of the head using
a special source, both a video system and an audio system require huge devices, and the system
becomes an expensive system.
[0010]
The video and audio reproduction apparatus according to the present invention has been made
in view of the above-described circumstances, and an object thereof is to provide a video and
audio reproduction apparatus capable of matching the sound image position and the video
position inexpensively while eliminating the need for special sources. I assume.
[0011]
10-05-2019
2
SUMMARY OF THE INVENTION In order to solve the above problems, the video and audio
reproduction apparatus according to the present invention causes the audio signal processing
unit of the audio reproduction means to localize the reproduced sound image outside the head
and to head the sound image direction. Signal processing is performed to fix the viewing
environment in a certain direction regardless of the rotational movement of the unit, and the
video signal processing unit of the video reproduction means performs the video perception
position according to the rotational movement of the head. Perform signal processing to fix on.
[0012]
Further, in the video / audio reproduction apparatus according to the present invention, in order
to solve the above problems, each output of two digital filters that convolute two systems of
impulse responses into a plurality of audio input signals in an audio signal processing unit of
audio reproduction means. A time difference or phase difference, or a level difference or
frequency characteristic is added so that the reproduction sound image is fixed in a fixed
direction based on the head rotation angle according to the head rotation movement, and the
video signal processing unit of the video reproduction means The input image signal is subjected
to signal processing for fixing the image perception position of the input image signal in a
predetermined direction of the viewing environment according to the head rotation angle.
[0013]
Further, in the video and audio reproduction apparatus according to the present invention, in
order to solve the above problems, the head detected by the rotation angle detection means in
the video and audio signal generation means disposed at a place other than the head mounted
part Based on the position information according to the rotational movement of the unit, the
audio signal and the video signal in which the viewing environment is fixed are output to the two
sounding bodies on the head mounted unit and the image display unit.
[0014]
BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the video and audio
reproduction apparatus according to the present invention will be described below with
reference to the drawings.
[0015]
First, in the first embodiment, an audio reproduction unit 2 that realizes localization outside the
head of a sound image to reproduce an input sound, and a video reproduction unit 10 that
realizes fixing of an image in a certain direction to reproduce an input image. And a rotational
angular velocity sensor 16 for detecting rotational movement of the head, for example, in a head
10-05-2019
3
band or helmet-like head mounting unit 15 integrally.
[0016]
The sound reproducing unit 2 localizes the input sound outside the head of the wearer of the
head mounted unit 15 according to the detection result of the rotational angular velocity sensor
16 and prevents the head from rotating in the sound image direction. The audio signal
processing unit 5 is provided to perform an operation to fix the sound image direction in a
certain direction of the viewing environment.
The audio signal input from the audio signal input terminal 3 is converted into a digital signal by
the A / D converter 4 and then supplied to the audio signal processing unit 5.
The output results from the audio signal processing unit 5 are output to D / A converters 6L and
6R corresponding to the left and right ears.
The audio signals converted into analog signals by the D / A converters 6L and 6R are disposed
at the positions of the left and right ears via the amplifiers 7L and 7R corresponding to the left
and right ears for headphone drive. The sound generator 8L for the ear and the sound generator
8R for the right ear are supplied.
[0017]
The video reproduction unit 10 includes a video signal processing unit 12 that fixes the input
video in a predetermined direction of the viewing environment of the wearer of the head
mounting unit 15 according to the detection result of the rotational angular velocity sensor 16.
The video signal input from the video signal input terminal 11 is supplied to the video signal
processing unit 12.
The video signal processed by the video signal processing unit 12 is a left-eye video display unit
14L and a right-eye video arranged near the front of the left and right eyes via the video display
10-05-2019
4
drive unit 13L and the video display drive unit 13R. It is supplied to the display unit 14R.
[0018]
The rotational angular velocity sensor 16 detects the rotational angular velocity of the head of
the wearer of the head mounted unit 15 and supplies the detection result to the microprocessor
17.
The microprocessor 17 calculates the rotational movement angle of the head mounted unit 15
from the front direction of the wearer, and supplies the calculation result to the audio signal
processing unit 5 and the video signal processing unit 12.
[0019]
Here, as shown in FIG. 2, the audio signal processing unit 5 uses the actual two sound sources of
the two-channel input audio signals received via the input terminals 201 and 202 and the head
mounting unit 15 Four digital filters 21L, 21R, 22L and 22R that perform convolution operation
on impulse responses using head related transfer functions that reach both ears, and a pair of
additions that add outputs of the same L and R polarity among these two channels Comparators
23L and 23R, a pair of time difference adding circuits 24L and 24R connected to the pair of
adders 23L and 23R, and a pair of level difference adding circuits 25L and 25R connected to the
pair of time difference adding circuits 24L and 24R. And the pair of time difference adding
circuits 24L and 24R and the pair of the pair according to the rotational movement angle
obtained through the rotational angular velocity sensor 16 and the microprocessor 17. The
increase or decrease direction of the time difference and level difference to be added by the bell
difference addition circuits 25L and 25R are reversed.
[0020]
Specifically, the audio signal processing unit 5 supplies audio signals to the sounding bodies 8L
and 8R assuming the state as shown in FIG.
That is, as the input sound signal, two channels from the sound source 301 and the sound source
302 are considered as those to be localized in the frontal range 180 °.
10-05-2019
5
[0021]
First, in the digital audio signals input from the input terminals 201 and 202, impulses
corresponding to the sound source 301 corresponding to localization in a certain direction in the
front in the initial state and impulses corresponding to head transfer functions from the sound
source 302 to both ears The response is convoluted by the digital filters 21L, 21R, 22L and 22R,
and the L side output is added by the adder 23L and output from the output terminal 26L via the
time difference adding circuit 24L and the level difference adding circuit 25L, R side output Are
added by the adder 23R and output from the output terminal 26R via the time difference adding
circuit 24R and the level difference adding circuit 25R.
[0022]
Here, HLl, HLr, HRl, and HRr as shown in FIG. 3 are considered as the head-related transfer
functions from the sound source 301 and the sound source 302 to the both ears l and r of the
listener M.
When the signals output from the L channel and R channel speakers are SL and SR, the impulse
response is digital filter 21 L so that SLHL1 + SRHR1 is supplied to the left ear 1 and SRHRr +
SLHLr to the right ear r. , 21R, 22L and 22R are given to the both ears of the head mounted unit
15 by the sounding bodies 8L and 8R.
[0023]
When the listener M moves the head to the left, for example, the left ear l moves away from the
sound sources 301 and 302, and the right ear r approaches the sound sources 301 and 302.
For this reason, time difference and level difference occur in the audio input signal reaching the
left ear l and the right ear r.
The pair of time difference adding circuits 24L and 24R and the pair of level difference adding
circuits 25L and 25R generate the time difference and the level difference.
10-05-2019
6
[0024]
The delay time added by the time difference adding circuit 24L for the L side is indicated by the
characteristic curve Tb of the one-dot chain line in the delay time characteristic of FIG. 4 and the
delay time added by the time difference adding circuit 24R for the R side is a diagram. It is
shown by the broken characteristic curve Ta of the delay time characteristic of four.
Characteristic curves Ta and Tb are curves having directions of increase and decrease which are
completely opposite to the direction of rotation of the head of the listener M.
As a result, the time change from the sound source to both ears similar to the case where the
listener M listens while rotating the head left and right from the sound source placed within the
range of 180 ° forward inputs from the input terminals 201 and 202 Will be added to the
signal.
[0025]
Further, the level difference added by the L level difference adding circuit 25L is indicated by the
characteristic curve La of the relative level characteristic of FIG. 5, and the level difference added
by the R side level difference adding circuit 25R. Is indicated by the characteristic curve Lb of the
relative level characteristic of the relative level characteristic of FIG.
This FIG. 5 shows the relative level from the state where the rotational position of the head is 0
°.
Characteristic curves La and Lb are curves having directions of increase and decrease which are
completely opposite to the direction of rotation of the head of the listener M.
That is, since the level change of the characteristic curve La is added in the level difference
adding circuit 25L and the level change of the characteristic curve Lb is added in the level
difference adding circuit 25R, the volume change similar to that of actually hearing the sound
source ahead is input terminal 201 and It is added to the input signal from 202.
10-05-2019
7
[0026]
From the above, the audio signal reproduced by the audio reproducing unit 2 changes in the
same manner as when listening to the actual sound while moving the head, and therefore
localizes the listening environment including the front in a fixed direction outside the head It will
be.
In addition, by using a plurality of audio signals of two or more channels, three-dimensional
sound field reproduction in which a sound image is localized at an arbitrary position can be
performed.
[0027]
Next, as shown in FIG. 6, the video signal processing unit 12 encodes the video input signal
received through the input terminal 31 into a digital signal by the A / D converter 32 and stores
it in the image memory 33, The display position of the image is processed to change according to
the rotation angle of the head.
[0028]
A storage address for storing video data, which is a coded video signal, in the image memory 33
is determined by the memory controller 37 in accordance with the rotational movement angle
calculated by the microprocessor 17.
Therefore, the image memory 33 takes in the video data as a part of the display area. In the
image memory 33 from which video data is not fetched, data representing black or another
constant color is stored. The D / A converter 34 converts the video data into an analog signal as a
video signal such that the entire image memory 33 is displayed. This video signal is supplied
from the output terminal 36 through the video output amplifier 35 to the video display drive
units 13L and 13R shown in FIG. Then, the image is displayed on the image display units 14L
and 14R. Therefore, the image data arrangement position on the image memory 33 changes
according to the rotation angle of the head, and is fixed in a certain direction of the viewing
environment on the image display units 14L and 14R arranged in the front vicinity of both eyes
You will see a video that looks like
10-05-2019
8
[0029]
Here, the display areas on the image display units 14L and 14R are configured as shown in FIG.
That is, the display area is 640 × 480 dots, and each dot corresponds to an independent storage
area of the image memory 33. Here, the storage capacity secured for each dot is determined by
the bit length in the case of handling as a digital signal, the presence or absence of color display,
and the like.
[0030]
In the above display area, the case of limiting display of video data with a resolution of 320 ×
200 dots is considered. First, let the coordinates of the upper left corner of the display area be (0,
0), and let the coordinates of the upper left corner of the limited display area in the initial state
be (x1, y1). When the head rotates to the left, the video position must move to the right on the
limited display area, that is, (x2, y1) where x2> x1 , in order for the video to stay at a certain
position in the viewing environment. It does not. Here, the movement amount x2-x1 is
determined by the rotation angle θ (rad) of the head and the distance r from the head rotation
center to the display device, and when θ is small, it is calculated by about x2−x1 = rθ. In order
to realize this, the memory controller 37 is an image memory corresponding to the area where
the image data coordinates on the display area are surrounded by (x2, y1), (x2, y1 + 200), (x2 +
320, y1 + 320), and (x2 + 320, y1). 33 Move the storage position up.
[0031]
As described above, the head-mounted video and audio reproduction apparatus 1 according to
the first embodiment of the present invention includes the audio signal processing unit 5 and the
video signal processing unit according to the rotational movement angle of the head calculated
by the microprocessor 17 Since the sound image localization position and the image position are
signal processed so as to stay in a fixed direction outside the head and in the viewing
environment, the sound image localization position and the image position overlap as in the prior
art. The position of the sound source and the actual sound image can be always matched on the
projected image. Furthermore, the synergetic effect of the image and the sound image makes it
possible to obtain clearer sound image localization than in the case of only sound, and to realize
realistic image and sound reproduction.
10-05-2019
9
[0032]
In the head-mounted type video and audio reproduction apparatus 1, the audio signal processing
unit 5 of the audio reproduction unit 2 may be configured as shown in FIG. 8, FIG. 10 and FIG.
[0033]
The audio signal processing unit 5 shown in FIG. 8 uses a pair of frequency characteristic control
circuits 27L and 27R at the output side of the pair of time difference addition circuits 24L and
24R instead of the pair of level difference addition circuits 25L and 25R shown in FIG. It
becomes connected.
[0034]
Here, the pair of frequency characteristic control circuits 27L and 27R apply frequency
characteristics as shown in FIG. 9 to the input signal in accordance with the rotation angle of the
head of the head mounted portion wearer, and control the frequency characteristics. It is a thing.
The head is fixed to the front on the front (shown as 0 °.
As shown by the solid line, the response is constant even if the frequency f increases as shown by
the solid line, but in the case of rotating 90 ° to the right and 90 ° to the left, for example, the
response difference increases as the frequency f increases. It occurs. When the head is turned 90
° to the right (shown as + 90 °), the response increases as the frequency f becomes higher, as
shown by the one-dot chain line. On the other hand, when the head is turned to the left by 90 °
(shown as -90 °), the response decreases as the frequency f increases, as indicated by the
broken line. Both are vertically symmetrical with respect to the response characteristic shown by
the solid line when the head is fixed in the front direction.
[0035]
Therefore, the audio signal processing unit 5 shown in FIG. 8 increases or decreases the time
difference to be added according to the head rotation angle by the pair of time difference
addition circuits 24L and 24R to the input signals from the two sound sources of the front two
channels. Since the change direction of the frequency characteristic controlled in accordance
with the head rotation angle is reversed by the pair of frequency characteristic control circuits
27L and 27R, the actual sound source for the forward sound signal is headed. Since it is possible
10-05-2019
10
to realize the same time difference between the ears as moving and listening and the frequency
characteristics, it is possible to realize good out-of-head sound image localization in all directions.
[0036]
Next, the audio signal processing unit 5 shown in FIG. 10 uses a pair of phase difference adding
circuits 28L and 28R instead of the pair of time difference adding circuits 24L and 24R shown in
FIG. 2 as a pair of level difference adding circuits 25L and 25R. It is connected to the input side.
[0037]
Here, the pair of phase difference addition circuits 28L and 28R apply phase differences
according to the phase change characteristics as shown in FIG. 11 to the input signal according
to the rotation angle of the head of the wearer.
With the head fixed to the front and the front (shown as 0 °.
In the case of), the phase difference θ is as shown by a solid line, but for example, when the
head is rotated 90 ° to the right and 90 ° to the left, the phase difference shifts to the left and
right. When the head is turned to the right by 90 ° (shown as + 90 °), the phase advances as
shown by a dot-and-dash line. On the other hand, when the head is turned 90 ° to the left
(shown as -90 °), the phase is delayed as shown by the broken line.
[0038]
Therefore, according to the head-mounted type video / audio reproduction device 1 using the
audio signal processing unit 5 shown in FIG. 10, the pair of phase difference addition circuits
28L and 28R gives the head input signals from the sound sources of the two front channels.
Since the increase and decrease direction of the phase difference to be added is reversely given
according to the part rotation angle, and the increase and decrease direction of the level
difference to be added according to the head rotation angle is given by the pair of level
difference addition circuits 25L and 25R. Since it is possible to reproduce the phase difference
characteristic between both ears and the level difference characteristic equivalent to listening to
the actual sound source while moving the head against the front sound signal, it is possible to
realize good out-of-head sound image localization in all directions. .
10-05-2019
11
[0039]
Next, the audio signal processing unit 5 shown in FIG. 12 is replaced by a pair of phase
difference adding circuits 28L and 28R in place of the pair of time difference adding circuits 24L
and 24R shown in FIG. Connected to the input side of.
[0040]
Here, the pair of phase difference adding circuits 28L and 28R are configured to set the phase
difference according to the phase change characteristic as shown in FIG. give.
[0041]
Further, the pair of frequency characteristic control circuits 27L and 27R apply frequency
characteristics as shown in FIG. 9 to the input signal in accordance with the rotation angle of the
head of the above-mentioned person wearing the apparatus 1 to control the frequency
characteristics. is there.
[0042]
Therefore, according to the head-mounted video / audio reproduction device 1 using the audio
signal processing unit 5 shown in FIG. 12, the input signals from the two sound sources in the
front two channels are paired with the phase difference addition circuits 28L and 28R. The
second pair of phase difference addition circuits 31L and 31R reversely applies the increase or
decrease direction of the phase difference to be added according to the head rotation angle, and
the first pair of frequency characteristic control circuits 28L and 28R, the second Since the
change direction of the frequency characteristic controlled according to the head rotation angle
is reversed by the pair of frequency characteristic control circuits 29L and 29R, the actual sound
source can be used for both the front audio signal and the rear audio signal. Since it is possible to
realize the phase difference between the two ears that is equivalent to listening while moving the
head, and the frequency characteristics, it is possible to realize good out-of-head sound image
localization in any direction.
[0043]
Here, the rotational angular velocity sensor 16 shown in FIG. 1 will be described.
The rotational angular velocity sensor 16 detects the rotational angular velocity of the head of
10-05-2019
12
the wearer wearing the head mounting unit 15.
In particular, in the head-mounted video / audio reproduction apparatus according to the first
embodiment, a piezoelectric vibration gyro apparatus 80 as shown in FIG. 13 is used as the
rotational angular velocity sensor 16.
The piezoelectric vibration gyro device 80 is a device that detects the swinging motion of a
moving body by using a piezoelectric element.
In FIG. 13, the vibrating piezoelectric element 81 made of a vibrating square pole of square cross
section is composed of various vibrating bodies.
Detection piezoelectric elements 82 and 83 and driving piezoelectric elements 84 and 85 are
attached to two opposing surfaces of the vibration piezoelectric element 81.
[0044]
A driving signal source 86 is connected to the driving piezoelectric elements 84 and 85 so as to
supply an alternating signal. The outputs of the detection piezoelectric elements 82 and 83 are
supplied to a differential amplifier 87. The differential output of the differential amplifier 87 and
the output of the driving signal source 86 are supplied to a multiplier or phase detector 88 for
multiplication or phase detection. The output of the multiplier or phase detector 88 is provided
to the microprocessor 17 shown in FIG.
[0045]
The piezoelectric gyro device 80 configured as described above operates as follows. First, when
an alternating signal of the natural vibration frequency of the vibration piezoelectric element 81
is applied to the drive piezoelectric elements 84 and 85, the vibration piezoelectric element 81 is
forcibly vibrated based on the illustrated vibration waveform. This vibration generates resonance
in a constant mode.
10-05-2019
13
[0046]
In this case, when no external force is applied, the detection piezoelectric elements 82 and 83
have no output, but when rotational force of angular velocity ω is applied to the vibration
piezoelectric element 81 in the axial direction, Coriolis force acts as a carrier wave. The
alternating signal for forced vibration is amplitude-modulated and detected as a detection signal.
The magnitude of the amplitude in this case is proportional to the angular velocity ω of the
rotation exerted on the shaft, and the direction of rotation corresponds to the direction of phase
shift of the detection signal with respect to the drive signal.
[0047]
Therefore, the product of the amplitude-modulated detection signal and the driving signal is
obtained, and the carrier wave component is removed by a band-limiting filter as a low-pass filter
to obtain a detection signal.
[0048]
The rotational angular velocity sensor 16 may be an analog angle detector 90 as shown in FIG.
The analog angle detector 90 is provided on the head mounting unit 15 to detect movement of
the head. In the analog angle detector 90, a light receiver 91 made of a light receiving element
whose resistance value changes according to the intensity of light such as a CDS or a photodiode
is attached to the central portion of the head mounting portion 15. A light emitting device 93
such as a light bulb or a light emitting diode is provided to face the light receiving device 91, and
the light emitting device 93 irradiates the light receiving device 91 with light having a
predetermined intensity.
[0049]
At this time, a movable shutter 92 is provided between the paths of the projected light of the
light emitter 93 so that the transmission of the projected light changes due to the rotation angle,
and the movable shutter 92 rotates with the magnetic needle 94. ing. Therefore, when a constant
current is supplied to the light receiver 91, the voltage at both ends of the light receiving element
of the light receiver 91 is an analog showing the movement of the head including the direction of
10-05-2019
14
the wearer of the head mounting portion 15 with reference to the north-south direction Output is
taken out.
[0050]
Also, the rotational angular velocity sensor 16 may be a digital angle detector 95 as shown in
FIG. The digital angle detector 95 is provided on the head mounting unit 15 and detects
movement of the head. In the digital angle detector 95, a rotary encoder 96 is provided at the
center of the head mounting portion 15 so that its input axis is vertical, and a magnetic needle
97 is provided on the input axis. Therefore, from the rotary encoder 96, an output indicating the
movement of the head including the direction of the wearer of the head mounting unit 15 is
extracted with reference to the north-south direction indicated by the magnetic needle 97.
[0051]
Further, the rotational angular velocity sensor 16 may calculate the rotational angle based on the
output ratio of at least two light intensity sensors provided on the head mounted unit 15 and the
light emitter placed in front of or around the sensor.
[0052]
Further, the rotational angular velocity sensor 16 is a microphone mounted at two distant places
on the head mounting unit 15 and reads a burst signal intermittently generated from an
ultrasonic oscillator placed in front of or around the periphery. The rotation angle may be
calculated from the time difference between the respective received signals.
[0053]
Next, a second embodiment will be described.
In the second embodiment, as shown in FIG. 16, an audio reproduction unit 42 for realizing an
external localization of a sound image to reproduce an input sound, and a fixation of an image in
a certain direction for an input image are realized. The head mounting unit 47 is integrally
provided with a video reproduction unit 48 to be reproduced and a rotational angular velocity
sensor 45 for detecting rotational movement of the head, and binocular parallax is used for the
image displayed on the video reproduction unit 48 It is a head-mounted type video and audio
10-05-2019
15
reproduction device 40 which is a stereoscopic video.
[0054]
The configuration of the audio reproduction unit 42 is the same as that of the audio reproduction
unit 10 according to the first embodiment.
That is, the audio signal processing unit 5 is configured as shown in FIG. 2, FIG. 8, FIG. 10 or FIG.
12, and the above-mentioned audio signal processing is applied to the input audio signal input
through the input terminal 43 to perform D / A conversion. And the sound generator 44L and
44R via an amplifier.
[0055]
As shown in FIG. 17, the image reproduction unit 48 is an image for the left eye in which
binocular parallax occurs, obtained by photographing the same subject 55 with the binocular
camera (for the left) 56L and the binocular camera (for the right) 56R. The signal and the video
signal for the right eye are input through the input terminals 49L and 49R. The video signal for
the left eye is supplied to the video signal processing unit 50L for the left eye, and processed
here so as to be fixed in a predetermined direction of the viewing environment according to the
detection result of the rotational angular velocity sensor 45. Also, the video signal for the right
eye is supplied to the video signal processing unit 50R for the right eye, and is processed here so
as to be fixed in a predetermined direction of the viewing environment according to the detection
result. That is, in the video signal processing unit 50L and the video signal processing unit 50R,
the display positions of the video signal for the left eye and the video signal for the right eye in
the video display unit 52L and the video display unit 52R according to the rotation angle of the
head. Process to change. When the head rotates to the right, the frame combining position of the
video signal for the left eye and the video signal for the right eye moves to the left in the video
display unit 52L and the video display unit 52R, and conversely, the head When rotating to the
left, the frame combining position moves to the right in the video display unit 52L and the video
display unit 52R. For this reason, the video appears to be staying in a certain direction in the
viewing environment. Also, the video signal is perceived as a stereoscopic image with a sense of
perspective because it captures binocular parallax as described above.
[0056]
10-05-2019
16
The video signal processing units 50L and 50R have the same configuration as that shown in FIG.
Further, the display areas of the image display units 52L and 52R have the same configuration as
that shown in FIG. The description is omitted here.
[0057]
As described above, the head-mounted video / audio reproduction apparatus 40 according to the
second embodiment of the present invention includes the audio signal processing unit and the
video signal processing unit 50L according to the rotational movement angle of the head
calculated by the microprocessor 46. And 50R to process the sound image localization position
and the image position where binocular parallax occurs to stay in a fixed direction outside the
head and in the viewing environment, the sound image localization position and the image
position overlap as in the prior art. It is possible to eliminate the sense of discomfort due to the
sound source, and to always match the positions of the sound source and the actual sound image
on the image projected three-dimensionally on the display. Furthermore, the synergetic effect of
the stereoscopic image and the sound image makes it possible to obtain clearer sound image
localization as compared with the case of only the sound, and to realize realistic stereoscopic
video and audio reproduction.
[0058]
Next, a third embodiment will be described. The third embodiment has a configuration similar to
that of the head-mounted video / audio reproduction device 1 shown in FIG. 1, but the video
signal processing unit 12 selectively outputs a part of the video signal. Video signal processing.
[0059]
That is, according to the head rotation angle calculated by the microprocessor 17 from the
rotation angular velocity detected by the rotation angular velocity sensor 16, whether the
content displayed on the image display units 14L and 14R is fixed in a certain direction of the
viewing environment To change.
[0060]
10-05-2019
17
The state of selection of such a video area will be described with reference to FIG.
After the video signal processing unit 12 outputs the display area 58 of the video signal content
entire area 57 to the video display units 14L and 14R in the initial state, the video signal
processing unit 12 is turned when the viewer's head is turned to the right. Sequentially output a
display area 59, which is an image matching the rotation angle, to the image display units 14L
and 15R. Conversely, when the head is turned to the left from this state, it heads in the direction
of the display area 58.
[0061]
Although the video signal processing unit 12 has the same configuration as that shown in FIG. 6
above, although the method of expanding the video data on the image memory 33 is a part of the
memory 33 in the first embodiment. On the other hand, the video data is stored in the entire area
of the image memory 33, and a part of the actual display area is cut out and displayed.
[0062]
The configuration and operation of the audio reproduction unit 2 are the same as those of the
audio reproduction unit 2 shown in FIG.
[0063]
As described above, since the video signal processing unit 12 keeps the display content in a
certain direction in the viewing environment, the third embodiment enables video and audio
reproduction in which the video image and the sound image direction coincide with each other.
[0064]
Next, a fourth embodiment will be described.
In the fourth embodiment, as shown in FIG. 19, the position information of the wearer of the
head-mounted unit 65 is input to the video and audio signal generator 61 provided separately
from the head-mounted unit 65, and the position information is provided. The head-mounted
video / audio reproduction device 60 outputs the video signal and the audio signal according to
the video / audio environment so as to be fixed.
10-05-2019
18
[0065]
The head rotation angle detected by the rotation angular velocity sensor 66 provided in the head
mounting unit 65 is input to the microprocessor 67 and converted into a position information
control signal here.
The position information control signal is supplied to the video and audio signal generator 61 via
the position information output terminal 68.
[0066]
The video / audio signal generation unit 61 knows the positional information of the wearer of the
head-mounted unit 65 according to the positional information control signal, and updates the
contents of the video signal and the audio signal.
Here, the video and audio signal generation unit 61 synthesizes and outputs the video signal by a
method such as computer graphics, stores the audio signal in a large scale memory in advance,
and selectively extracts it from the large scale memory. Output. Therefore, the video / audio
signal generation unit 61 displays the video signal synthesized by the method such as the
computer graphic updated according to the head position of the head mounting unit 65 through
the video input terminal 62. It supplies to drive part 63L and 63R. Further, the video / audio
signal generation unit 61 reproduces the above-mentioned audio signal selectively extracted
from the large-scale memory updated according to the head position of the head-mounted unit
65 wearing user via the audio input terminal 69. It supplies to the part 70.
[0067]
The sound reproduction unit 70 performs sound signal processing similar to that performed by
the sound reproduction unit 1 of FIG. 1 on the sound signal, and a sound generator 71 L
disposed to form a headphone in the head mounted unit 65 And supply to 71R. Therefore, the
audio signal is processed so that the sound image is fixed at a fixed position outside the head
according to the movement of the viewer's head. Further, by using a plurality of audio signals,
10-05-2019
19
three-dimensional sound field reproduction in which a plurality of sound images are localized at
an arbitrary position can be performed.
[0068]
On the other hand, the video display drive units 63L and 63R, to which the video signal is input
through the video input terminal 62, are disposed so that the video signal is in the head mounted
unit 65 and located near both eyes. It supplies to the image display parts 64L and 64R. The
image display units 64L and 64R display the image signal such that the content itself is fixed in
the viewing environment. That is, when the head is rotated to the right, the image is such that the
image is panned to the right, and conversely, when the head is rotated to the left, the image is
panned to the left and moved . For this reason, the display content seems to be staying in a
certain direction in the viewing environment.
[0069]
As described above, the head-mounted video and audio reproduction apparatus 60 according to
the fourth embodiment can be configured such that the actual sound image is localized to the
sound generation source on the video including the sense of closeness in the video and sound
field. Extremely realistic video and audio reproduction becomes possible.
[0070]
The fifth embodiment will now be described.
Similar to the third embodiment, the fifth embodiment has a configuration like the head-mounted
type video and audio reproduction device 1 shown in FIG. 1, and the video signal processing unit
12 is an example of a video signal. The video signal processing is performed to selectively output
the part, and when the display area reaches the limit of the video signal content, the video and
sound image localization positions are synchronized with the head movement and rotated with
respect to the viewing environment.
[0071]
10-05-2019
20
That is, after the video signal processing unit 12 outputs the display area 58 of the video signal
content entire area 57 to the video display units 14L and 14R in the initial state, the video signal
processing is performed when the viewer's head is rotated to the right The unit 12 continuously
outputs the display area 59, which is an image matching the rotation angle, to the image display
units 14L and 15R. Conversely, when the head is turned to the left from this state, it heads in the
direction of the display area 58.
[0072]
Here, as shown in FIG. 20, when the display area 85 reaches the limit of the entire area 57 of the
video signal content, the video and sound image localization positions are synchronized with the
movement of the head at that time to the viewing environment. When the head is rotated in the
reverse direction in this state, both video and sound images are fixed to the viewing environment
as long as the video signal content entire area 57 permits. Be done.
[0073]
As described above, when the viewer usually watches software such as an existing movie by
wearing this fifth embodiment, the video and audio need not be fixed within the entire range, for
example, as shown in FIG. If the video position is fixed within a predetermined range W of the
front F of the viewer M as shown in FIG. 21, the video position and the sound image position will
be fixed within a practically sufficient range. The displayed content appears to be in a fixed
direction in the viewing environment, and video and audio reproduction in which the video image
and the sound image direction coincide can be achieved.
In addition, since the time difference and level difference between both ears due to the
movement of the head with respect to the audio signal are added as well as listening to the real
sound source, the sound image can be localized in the state including the head and the forward
direction.
[0074]
The voice input signal may be supplied to the voice reproduction means on the head-mounted
unit wirelessly. In addition, the video input signal may be supplied to the video reproduction
means on the head-mounted unit wirelessly.
10-05-2019
21
[0075]
According to the present invention, the audio / video reproduction apparatus according to the
present invention causes the audio signal processing unit of the audio reproduction means to
localize the reproduced sound image outside the head and the sound image direction is a fixed
direction of the viewing environment regardless of the rotational movement of the head. Signal
processing to fix the image perceptual position in a certain direction of the viewing environment
according to the rotational movement of the head according to the rotational motion of the head.
The sound image can be localized outside the head, particularly in the forward direction where
localization is conventionally considered difficult. In addition, since the image position is fixed at
a predetermined position, the sound image localization position and the image position overlap,
so that the sense of discomfort due to the mismatch between the both can be eliminated.
Furthermore, the synergetic effect of the image and the sound image makes it possible to obtain
clearer sound image localization than in the case of only sound, and to realize realistic image and
sound reproduction. Furthermore, since sound image out-of-head localization processing can be
realized by signal processing with a simple configuration, a low-cost apparatus with excellent
portability can be configured in a small size.
[0076]
In addition, if binocular parallax is added to the video signal to be input, it is possible to
reproduce a stereoscopic image with a long distance, and as in the prior art, the sound image
localization position and the video position overlap and the discomfort due to the mismatch
between both is eliminated. It is possible to always match the positions of the sound source and
the actual sound image on the image projected three-dimensionally on the display. Furthermore,
the synergetic effect of the stereoscopic image and the sound image makes it possible to obtain
clearer sound image localization as compared with the case of only the sound, and to realize
realistic stereoscopic video and audio reproduction.
[0077]
In addition, since the video reproduction means can selectively display a part of the video signal,
it is possible to view the sound image localization position and the video position without
reducing the amount of video viewed by the viewer for the movement of a certain range of the
head. Fixation to the environment can be realized.
10-05-2019
22
[0078]
Further, according to the present invention, for example, the present invention is applied to a
system capable of synthesizing an image in real time, such as computer graphics, and the
contents of the image content itself can be updated and displayed according to the rotational
movement of the head. Fixation of the sound image localization position and the image quality to
the viewing environment can be realized without reducing the amount of image viewed by the
viewer with respect to the movement of the range.
[0079]
In addition, the sound reproduction means and the video reproduction means fix the sound
image direction in a certain direction of the viewing environment only within a certain range
centered on the front direction according to the movement of the head. Since the voice direction
can be moved according to the movement, the range in which the sound image direction is fixed
to a certain direction of the viewing environment is limited even when viewing an image whose
display content does not change according to the head movement. Therefore, the direction of the
image and the direction of the sound image do not largely deviate, and the forward localization of
the sound image by dynamic sound signal processing is enabled, and a wide image display range
such as movie viewing is not required. Even with video and audio sources to be watched with, it
makes it easy to reproduce video and audio with sufficient reality and realism.
[0080]
Further, in the present invention, the head mounting unit is provided with the rotation detection
means and the sound reproduction means, and the movement of the head is detected, and the
image direction and the sound image direction are constant directions of the viewing
environment only within a certain range centered on the front direction. Changes as if fixed, and
if it exceeds the range, it moves in both the image direction and the sound image direction
according to the movement of the head, so that the display content changes according to the
movement of the head Even when the information capacity is limited, in the case where the
rotational movement range of the head is limited, the image direction and the sound image
direction can be fixed in a certain direction of the viewing environment.
In addition, it enables the forward localization of the sound image by dynamic sound signal
processing, and there is sufficient reality and presence for a system that allows viewing in a state
with few normal movements without requiring a wide image display range such as movie
viewing. Video and audio reproduction can be easily configured.
10-05-2019
23
[0081]
In addition, by incorporating the audio reproduction means in the head-mounted unit, the
audiovisual signal having a conventional ordinary audio signal source is directly input without
having special recording such as binaural recording, etc. to have localization outside the head It
enables immersive voice reproduction and enables a portable virtual reality system to be
miniaturized.
[0082]
Also, by providing the position detection means in the head mounting unit instead of or
simultaneously with the rotation angle detection means, not only control when the wearer rotates
but also an image and sound image when the position is moved in the viewing environment
Control is also possible, and a more realistic virtual reality system can be configured compactly.
[0083]
Brief description of the drawings
[0084]
1 is a block diagram showing a schematic configuration of a first embodiment of a video and
audio reproduction apparatus according to the present invention.
[0085]
2 is a block diagram showing a detailed configuration of the audio signal processing unit used in
the first embodiment.
[0086]
3 is a schematic view showing a listening environment assumed by the audio signal processing
unit used in the first embodiment.
[0087]
4 is a characteristic diagram of the delay time added by the time difference adding circuit of the
audio signal processing unit used in the first embodiment.
[0088]
10-05-2019
24
5 is a characteristic diagram of the level difference added by the level difference adding circuit of
the audio signal processing unit used in the first embodiment.
[0089]
6 is a block diagram showing a detailed configuration of a video signal processing unit used in
the first embodiment.
[0090]
7 is a schematic view of a display area of the video reproduction unit used in the first
embodiment.
[0091]
FIG. 8 is a block diagram in the case where the audio signal processing unit uses a time
difference adding circuit and a frequency characteristic control circuit.
[0092]
FIG. 9 is a characteristic diagram of frequency control performed by the frequency characteristic
control circuit.
[0093]
FIG. 10 is a block diagram when the audio signal processing unit uses a phase difference adding
circuit and a level difference adding circuit.
[0094]
11 is a change characteristic diagram of the phase difference added by the phase difference
adding circuit.
[0095]
12 is a block diagram in the case where the audio signal processing unit uses a phase difference
adding circuit and a frequency characteristic control circuit.
[0096]
13 is a schematic block diagram of a pressure-transfer vibration gyro device applicable to the
10-05-2019
25
rotational angular velocity sensor used in the first embodiment.
[0097]
14 is a schematic configuration diagram of an analog angle detector that can be applied to the
rotational angular velocity sensor used in the first embodiment.
[0098]
FIG. 15 is a schematic block diagram of a digital angle detector applicable to the rotational
angular velocity sensor used in the first embodiment.
[0099]
16 is a block diagram showing a schematic configuration of a second embodiment of the video
and audio reproduction apparatus according to the present invention.
[0100]
FIG. 17 is a schematic view for explaining a video signal in which binocular parallax occurs which
is input in the second embodiment.
[0101]
18 is a schematic diagram for explaining the operation of the video reproduction unit of the third
embodiment.
[0102]
19 is a schematic diagram for explaining a fourth embodiment of the video and audio
reproduction apparatus according to the present invention.
[0103]
FIG. 20 is a schematic view for explaining the operation of the video reproduction unit of the fifth
embodiment of the video and audio reproduction device according to the present invention.
[0104]
21 is a schematic diagram for explaining the operation of the fifth embodiment.
10-05-2019
26
[0105]
FIG. 22 is a schematic view of a conventional head-mounted type video and audio reproduction
apparatus.
[0106]
23 is a block diagram of a conventional video and audio reproduction apparatus.
[0107]
Explanation of sign
[0108]
Reference Signs List 1 head-mounted type video and audio reproduction device 2 audio
reproduction unit 5 audio signal processing unit 10 video reproduction unit 12 video signal
processing unit 14 video display unit 15 head mounting unit 16 rotational angular velocity
sensor 17 microprocessor
10-05-2019
27

Download Report