Patent Translate Powered by EPO and Google Notice This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate, complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or financial decisions, should not be based on machine-translation output. DESCRIPTION JPH0993700 [0001] BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video and audio reproduction apparatus capable of viewing video and audio by being worn on the head. [0002] 2. Description of the Related Art Conventionally, reproduction of a sense of reality in a video and audio reproduction system has mainly been the enlargement of a system by large screen video and multi-speaker reproduction. However, as shown in FIG. The video / audio reproduction apparatus 100 has been used in which personalization and miniaturization are achieved by the head-mounted type video display unit 101 and the audio reproduction unit 102 added thereto. [0003] As shown in FIG. 23, the video display unit 101 is driven by the drive circuit 104 supplied with a video input signal from the video input terminal 103 to display a video. [0004] Similarly, as shown in the figure, the audio reproduction unit 102 amplifies the audio input signal input from the audio input terminal 105 by the amplifiers 106L and 106R, and supplies it to the left-ear sound producing body 107L and the right-ear sound producing body 107R. , Giving a voice to the wearer. 10-05-2019 1 [0005] Here, since the sound reproducing unit 102 localizes the sound image in the head of the wearer, the sound reproducing unit 102 is very unnatural in the sense of reproducing the sense of reality without being matched with the image displayed by the image display unit 101. [0006] Therefore, the video and audio are updated in real time according to the movement of the body, and further, in the audio, an attempt is made to reproduce a near-real viewing environment by convoluting, for example, with a digital signal processing unit. ing. [0007] By the way, in the above-mentioned attempts, a great number of signal processing is required especially in speech processing, and it can not be made large enough to be worn on the head by itself, and movement The detection means also had to be expensive and was not practical. [0008] Also, even if it is possible to realize the sound field feeling close to reality in the sound by realizing the localization outside the head of the sound image, the video system moves in the viewing environment following the movement of the head as in the conventional case. A large deviation occurs between the sound image position and the video position, resulting in unnatural video and audio reproduction. [0009] Further, even when the video contents are updated according to the movement of the head using a special source, both a video system and an audio system require huge devices, and the system becomes an expensive system. [0010] The video and audio reproduction apparatus according to the present invention has been made in view of the above-described circumstances, and an object thereof is to provide a video and audio reproduction apparatus capable of matching the sound image position and the video position inexpensively while eliminating the need for special sources. I assume. [0011] 10-05-2019 2 SUMMARY OF THE INVENTION In order to solve the above problems, the video and audio reproduction apparatus according to the present invention causes the audio signal processing unit of the audio reproduction means to localize the reproduced sound image outside the head and to head the sound image direction. Signal processing is performed to fix the viewing environment in a certain direction regardless of the rotational movement of the unit, and the video signal processing unit of the video reproduction means performs the video perception position according to the rotational movement of the head. Perform signal processing to fix on. [0012] Further, in the video / audio reproduction apparatus according to the present invention, in order to solve the above problems, each output of two digital filters that convolute two systems of impulse responses into a plurality of audio input signals in an audio signal processing unit of audio reproduction means. A time difference or phase difference, or a level difference or frequency characteristic is added so that the reproduction sound image is fixed in a fixed direction based on the head rotation angle according to the head rotation movement, and the video signal processing unit of the video reproduction means The input image signal is subjected to signal processing for fixing the image perception position of the input image signal in a predetermined direction of the viewing environment according to the head rotation angle. [0013] Further, in the video and audio reproduction apparatus according to the present invention, in order to solve the above problems, the head detected by the rotation angle detection means in the video and audio signal generation means disposed at a place other than the head mounted part Based on the position information according to the rotational movement of the unit, the audio signal and the video signal in which the viewing environment is fixed are output to the two sounding bodies on the head mounted unit and the image display unit. [0014] BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the video and audio reproduction apparatus according to the present invention will be described below with reference to the drawings. [0015] First, in the first embodiment, an audio reproduction unit 2 that realizes localization outside the head of a sound image to reproduce an input sound, and a video reproduction unit 10 that realizes fixing of an image in a certain direction to reproduce an input image. And a rotational angular velocity sensor 16 for detecting rotational movement of the head, for example, in a head 10-05-2019 3 band or helmet-like head mounting unit 15 integrally. [0016] The sound reproducing unit 2 localizes the input sound outside the head of the wearer of the head mounted unit 15 according to the detection result of the rotational angular velocity sensor 16 and prevents the head from rotating in the sound image direction. The audio signal processing unit 5 is provided to perform an operation to fix the sound image direction in a certain direction of the viewing environment. The audio signal input from the audio signal input terminal 3 is converted into a digital signal by the A / D converter 4 and then supplied to the audio signal processing unit 5. The output results from the audio signal processing unit 5 are output to D / A converters 6L and 6R corresponding to the left and right ears. The audio signals converted into analog signals by the D / A converters 6L and 6R are disposed at the positions of the left and right ears via the amplifiers 7L and 7R corresponding to the left and right ears for headphone drive. The sound generator 8L for the ear and the sound generator 8R for the right ear are supplied. [0017] The video reproduction unit 10 includes a video signal processing unit 12 that fixes the input video in a predetermined direction of the viewing environment of the wearer of the head mounting unit 15 according to the detection result of the rotational angular velocity sensor 16. The video signal input from the video signal input terminal 11 is supplied to the video signal processing unit 12. The video signal processed by the video signal processing unit 12 is a left-eye video display unit 14L and a right-eye video arranged near the front of the left and right eyes via the video display 10-05-2019 4 drive unit 13L and the video display drive unit 13R. It is supplied to the display unit 14R. [0018] The rotational angular velocity sensor 16 detects the rotational angular velocity of the head of the wearer of the head mounted unit 15 and supplies the detection result to the microprocessor 17. The microprocessor 17 calculates the rotational movement angle of the head mounted unit 15 from the front direction of the wearer, and supplies the calculation result to the audio signal processing unit 5 and the video signal processing unit 12. [0019] Here, as shown in FIG. 2, the audio signal processing unit 5 uses the actual two sound sources of the two-channel input audio signals received via the input terminals 201 and 202 and the head mounting unit 15 Four digital filters 21L, 21R, 22L and 22R that perform convolution operation on impulse responses using head related transfer functions that reach both ears, and a pair of additions that add outputs of the same L and R polarity among these two channels Comparators 23L and 23R, a pair of time difference adding circuits 24L and 24R connected to the pair of adders 23L and 23R, and a pair of level difference adding circuits 25L and 25R connected to the pair of time difference adding circuits 24L and 24R. And the pair of time difference adding circuits 24L and 24R and the pair of the pair according to the rotational movement angle obtained through the rotational angular velocity sensor 16 and the microprocessor 17. The increase or decrease direction of the time difference and level difference to be added by the bell difference addition circuits 25L and 25R are reversed. [0020] Specifically, the audio signal processing unit 5 supplies audio signals to the sounding bodies 8L and 8R assuming the state as shown in FIG. That is, as the input sound signal, two channels from the sound source 301 and the sound source 302 are considered as those to be localized in the frontal range 180 °. 10-05-2019 5 [0021] First, in the digital audio signals input from the input terminals 201 and 202, impulses corresponding to the sound source 301 corresponding to localization in a certain direction in the front in the initial state and impulses corresponding to head transfer functions from the sound source 302 to both ears The response is convoluted by the digital filters 21L, 21R, 22L and 22R, and the L side output is added by the adder 23L and output from the output terminal 26L via the time difference adding circuit 24L and the level difference adding circuit 25L, R side output Are added by the adder 23R and output from the output terminal 26R via the time difference adding circuit 24R and the level difference adding circuit 25R. [0022] Here, HLl, HLr, HRl, and HRr as shown in FIG. 3 are considered as the head-related transfer functions from the sound source 301 and the sound source 302 to the both ears l and r of the listener M. When the signals output from the L channel and R channel speakers are SL and SR, the impulse response is digital filter 21 L so that SLHL1 + SRHR1 is supplied to the left ear 1 and SRHRr + SLHLr to the right ear r. , 21R, 22L and 22R are given to the both ears of the head mounted unit 15 by the sounding bodies 8L and 8R. [0023] When the listener M moves the head to the left, for example, the left ear l moves away from the sound sources 301 and 302, and the right ear r approaches the sound sources 301 and 302. For this reason, time difference and level difference occur in the audio input signal reaching the left ear l and the right ear r. The pair of time difference adding circuits 24L and 24R and the pair of level difference adding circuits 25L and 25R generate the time difference and the level difference. 10-05-2019 6 [0024] The delay time added by the time difference adding circuit 24L for the L side is indicated by the characteristic curve Tb of the one-dot chain line in the delay time characteristic of FIG. 4 and the delay time added by the time difference adding circuit 24R for the R side is a diagram. It is shown by the broken characteristic curve Ta of the delay time characteristic of four. Characteristic curves Ta and Tb are curves having directions of increase and decrease which are completely opposite to the direction of rotation of the head of the listener M. As a result, the time change from the sound source to both ears similar to the case where the listener M listens while rotating the head left and right from the sound source placed within the range of 180 ° forward inputs from the input terminals 201 and 202 Will be added to the signal. [0025] Further, the level difference added by the L level difference adding circuit 25L is indicated by the characteristic curve La of the relative level characteristic of FIG. 5, and the level difference added by the R side level difference adding circuit 25R. Is indicated by the characteristic curve Lb of the relative level characteristic of the relative level characteristic of FIG. This FIG. 5 shows the relative level from the state where the rotational position of the head is 0 °. Characteristic curves La and Lb are curves having directions of increase and decrease which are completely opposite to the direction of rotation of the head of the listener M. That is, since the level change of the characteristic curve La is added in the level difference adding circuit 25L and the level change of the characteristic curve Lb is added in the level difference adding circuit 25R, the volume change similar to that of actually hearing the sound source ahead is input terminal 201 and It is added to the input signal from 202. 10-05-2019 7 [0026] From the above, the audio signal reproduced by the audio reproducing unit 2 changes in the same manner as when listening to the actual sound while moving the head, and therefore localizes the listening environment including the front in a fixed direction outside the head It will be. In addition, by using a plurality of audio signals of two or more channels, three-dimensional sound field reproduction in which a sound image is localized at an arbitrary position can be performed. [0027] Next, as shown in FIG. 6, the video signal processing unit 12 encodes the video input signal received through the input terminal 31 into a digital signal by the A / D converter 32 and stores it in the image memory 33, The display position of the image is processed to change according to the rotation angle of the head. [0028] A storage address for storing video data, which is a coded video signal, in the image memory 33 is determined by the memory controller 37 in accordance with the rotational movement angle calculated by the microprocessor 17. Therefore, the image memory 33 takes in the video data as a part of the display area. In the image memory 33 from which video data is not fetched, data representing black or another constant color is stored. The D / A converter 34 converts the video data into an analog signal as a video signal such that the entire image memory 33 is displayed. This video signal is supplied from the output terminal 36 through the video output amplifier 35 to the video display drive units 13L and 13R shown in FIG. Then, the image is displayed on the image display units 14L and 14R. Therefore, the image data arrangement position on the image memory 33 changes according to the rotation angle of the head, and is fixed in a certain direction of the viewing environment on the image display units 14L and 14R arranged in the front vicinity of both eyes You will see a video that looks like 10-05-2019 8 [0029] Here, the display areas on the image display units 14L and 14R are configured as shown in FIG. That is, the display area is 640 × 480 dots, and each dot corresponds to an independent storage area of the image memory 33. Here, the storage capacity secured for each dot is determined by the bit length in the case of handling as a digital signal, the presence or absence of color display, and the like. [0030] In the above display area, the case of limiting display of video data with a resolution of 320 × 200 dots is considered. First, let the coordinates of the upper left corner of the display area be (0, 0), and let the coordinates of the upper left corner of the limited display area in the initial state be (x1, y1). When the head rotates to the left, the video position must move to the right on the limited display area, that is, (x2, y1) where x2> x1 , in order for the video to stay at a certain position in the viewing environment. It does not. Here, the movement amount x2-x1 is determined by the rotation angle θ (rad) of the head and the distance r from the head rotation center to the display device, and when θ is small, it is calculated by about x2−x1 = rθ. In order to realize this, the memory controller 37 is an image memory corresponding to the area where the image data coordinates on the display area are surrounded by (x2, y1), (x2, y1 + 200), (x2 + 320, y1 + 320), and (x2 + 320, y1). 33 Move the storage position up. [0031] As described above, the head-mounted video and audio reproduction apparatus 1 according to the first embodiment of the present invention includes the audio signal processing unit 5 and the video signal processing unit according to the rotational movement angle of the head calculated by the microprocessor 17 Since the sound image localization position and the image position are signal processed so as to stay in a fixed direction outside the head and in the viewing environment, the sound image localization position and the image position overlap as in the prior art. The position of the sound source and the actual sound image can be always matched on the projected image. Furthermore, the synergetic effect of the image and the sound image makes it possible to obtain clearer sound image localization than in the case of only sound, and to realize realistic image and sound reproduction. 10-05-2019 9 [0032] In the head-mounted type video and audio reproduction apparatus 1, the audio signal processing unit 5 of the audio reproduction unit 2 may be configured as shown in FIG. 8, FIG. 10 and FIG. [0033] The audio signal processing unit 5 shown in FIG. 8 uses a pair of frequency characteristic control circuits 27L and 27R at the output side of the pair of time difference addition circuits 24L and 24R instead of the pair of level difference addition circuits 25L and 25R shown in FIG. It becomes connected. [0034] Here, the pair of frequency characteristic control circuits 27L and 27R apply frequency characteristics as shown in FIG. 9 to the input signal in accordance with the rotation angle of the head of the head mounted portion wearer, and control the frequency characteristics. It is a thing. The head is fixed to the front on the front (shown as 0 °. As shown by the solid line, the response is constant even if the frequency f increases as shown by the solid line, but in the case of rotating 90 ° to the right and 90 ° to the left, for example, the response difference increases as the frequency f increases. It occurs. When the head is turned 90 ° to the right (shown as + 90 °), the response increases as the frequency f becomes higher, as shown by the one-dot chain line. On the other hand, when the head is turned to the left by 90 ° (shown as -90 °), the response decreases as the frequency f increases, as indicated by the broken line. Both are vertically symmetrical with respect to the response characteristic shown by the solid line when the head is fixed in the front direction. [0035] Therefore, the audio signal processing unit 5 shown in FIG. 8 increases or decreases the time difference to be added according to the head rotation angle by the pair of time difference addition circuits 24L and 24R to the input signals from the two sound sources of the front two channels. Since the change direction of the frequency characteristic controlled in accordance with the head rotation angle is reversed by the pair of frequency characteristic control circuits 27L and 27R, the actual sound source for the forward sound signal is headed. Since it is possible 10-05-2019 10 to realize the same time difference between the ears as moving and listening and the frequency characteristics, it is possible to realize good out-of-head sound image localization in all directions. [0036] Next, the audio signal processing unit 5 shown in FIG. 10 uses a pair of phase difference adding circuits 28L and 28R instead of the pair of time difference adding circuits 24L and 24R shown in FIG. 2 as a pair of level difference adding circuits 25L and 25R. It is connected to the input side. [0037] Here, the pair of phase difference addition circuits 28L and 28R apply phase differences according to the phase change characteristics as shown in FIG. 11 to the input signal according to the rotation angle of the head of the wearer. With the head fixed to the front and the front (shown as 0 °. In the case of), the phase difference θ is as shown by a solid line, but for example, when the head is rotated 90 ° to the right and 90 ° to the left, the phase difference shifts to the left and right. When the head is turned to the right by 90 ° (shown as + 90 °), the phase advances as shown by a dot-and-dash line. On the other hand, when the head is turned 90 ° to the left (shown as -90 °), the phase is delayed as shown by the broken line. [0038] Therefore, according to the head-mounted type video / audio reproduction device 1 using the audio signal processing unit 5 shown in FIG. 10, the pair of phase difference addition circuits 28L and 28R gives the head input signals from the sound sources of the two front channels. Since the increase and decrease direction of the phase difference to be added is reversely given according to the part rotation angle, and the increase and decrease direction of the level difference to be added according to the head rotation angle is given by the pair of level difference addition circuits 25L and 25R. Since it is possible to reproduce the phase difference characteristic between both ears and the level difference characteristic equivalent to listening to the actual sound source while moving the head against the front sound signal, it is possible to realize good out-of-head sound image localization in all directions. . 10-05-2019 11 [0039] Next, the audio signal processing unit 5 shown in FIG. 12 is replaced by a pair of phase difference adding circuits 28L and 28R in place of the pair of time difference adding circuits 24L and 24R shown in FIG. Connected to the input side of. [0040] Here, the pair of phase difference adding circuits 28L and 28R are configured to set the phase difference according to the phase change characteristic as shown in FIG. give. [0041] Further, the pair of frequency characteristic control circuits 27L and 27R apply frequency characteristics as shown in FIG. 9 to the input signal in accordance with the rotation angle of the head of the above-mentioned person wearing the apparatus 1 to control the frequency characteristics. is there. [0042] Therefore, according to the head-mounted video / audio reproduction device 1 using the audio signal processing unit 5 shown in FIG. 12, the input signals from the two sound sources in the front two channels are paired with the phase difference addition circuits 28L and 28R. The second pair of phase difference addition circuits 31L and 31R reversely applies the increase or decrease direction of the phase difference to be added according to the head rotation angle, and the first pair of frequency characteristic control circuits 28L and 28R, the second Since the change direction of the frequency characteristic controlled according to the head rotation angle is reversed by the pair of frequency characteristic control circuits 29L and 29R, the actual sound source can be used for both the front audio signal and the rear audio signal. Since it is possible to realize the phase difference between the two ears that is equivalent to listening while moving the head, and the frequency characteristics, it is possible to realize good out-of-head sound image localization in any direction. [0043] Here, the rotational angular velocity sensor 16 shown in FIG. 1 will be described. The rotational angular velocity sensor 16 detects the rotational angular velocity of the head of 10-05-2019 12 the wearer wearing the head mounting unit 15. In particular, in the head-mounted video / audio reproduction apparatus according to the first embodiment, a piezoelectric vibration gyro apparatus 80 as shown in FIG. 13 is used as the rotational angular velocity sensor 16. The piezoelectric vibration gyro device 80 is a device that detects the swinging motion of a moving body by using a piezoelectric element. In FIG. 13, the vibrating piezoelectric element 81 made of a vibrating square pole of square cross section is composed of various vibrating bodies. Detection piezoelectric elements 82 and 83 and driving piezoelectric elements 84 and 85 are attached to two opposing surfaces of the vibration piezoelectric element 81. [0044] A driving signal source 86 is connected to the driving piezoelectric elements 84 and 85 so as to supply an alternating signal. The outputs of the detection piezoelectric elements 82 and 83 are supplied to a differential amplifier 87. The differential output of the differential amplifier 87 and the output of the driving signal source 86 are supplied to a multiplier or phase detector 88 for multiplication or phase detection. The output of the multiplier or phase detector 88 is provided to the microprocessor 17 shown in FIG. [0045] The piezoelectric gyro device 80 configured as described above operates as follows. First, when an alternating signal of the natural vibration frequency of the vibration piezoelectric element 81 is applied to the drive piezoelectric elements 84 and 85, the vibration piezoelectric element 81 is forcibly vibrated based on the illustrated vibration waveform. This vibration generates resonance in a constant mode. 10-05-2019 13 [0046] In this case, when no external force is applied, the detection piezoelectric elements 82 and 83 have no output, but when rotational force of angular velocity ω is applied to the vibration piezoelectric element 81 in the axial direction, Coriolis force acts as a carrier wave. The alternating signal for forced vibration is amplitude-modulated and detected as a detection signal. The magnitude of the amplitude in this case is proportional to the angular velocity ω of the rotation exerted on the shaft, and the direction of rotation corresponds to the direction of phase shift of the detection signal with respect to the drive signal. [0047] Therefore, the product of the amplitude-modulated detection signal and the driving signal is obtained, and the carrier wave component is removed by a band-limiting filter as a low-pass filter to obtain a detection signal. [0048] The rotational angular velocity sensor 16 may be an analog angle detector 90 as shown in FIG. The analog angle detector 90 is provided on the head mounting unit 15 to detect movement of the head. In the analog angle detector 90, a light receiver 91 made of a light receiving element whose resistance value changes according to the intensity of light such as a CDS or a photodiode is attached to the central portion of the head mounting portion 15. A light emitting device 93 such as a light bulb or a light emitting diode is provided to face the light receiving device 91, and the light emitting device 93 irradiates the light receiving device 91 with light having a predetermined intensity. [0049] At this time, a movable shutter 92 is provided between the paths of the projected light of the light emitter 93 so that the transmission of the projected light changes due to the rotation angle, and the movable shutter 92 rotates with the magnetic needle 94. ing. Therefore, when a constant current is supplied to the light receiver 91, the voltage at both ends of the light receiving element of the light receiver 91 is an analog showing the movement of the head including the direction of 10-05-2019 14 the wearer of the head mounting portion 15 with reference to the north-south direction Output is taken out. [0050] Also, the rotational angular velocity sensor 16 may be a digital angle detector 95 as shown in FIG. The digital angle detector 95 is provided on the head mounting unit 15 and detects movement of the head. In the digital angle detector 95, a rotary encoder 96 is provided at the center of the head mounting portion 15 so that its input axis is vertical, and a magnetic needle 97 is provided on the input axis. Therefore, from the rotary encoder 96, an output indicating the movement of the head including the direction of the wearer of the head mounting unit 15 is extracted with reference to the north-south direction indicated by the magnetic needle 97. [0051] Further, the rotational angular velocity sensor 16 may calculate the rotational angle based on the output ratio of at least two light intensity sensors provided on the head mounted unit 15 and the light emitter placed in front of or around the sensor. [0052] Further, the rotational angular velocity sensor 16 is a microphone mounted at two distant places on the head mounting unit 15 and reads a burst signal intermittently generated from an ultrasonic oscillator placed in front of or around the periphery. The rotation angle may be calculated from the time difference between the respective received signals. [0053] Next, a second embodiment will be described. In the second embodiment, as shown in FIG. 16, an audio reproduction unit 42 for realizing an external localization of a sound image to reproduce an input sound, and a fixation of an image in a certain direction for an input image are realized. The head mounting unit 47 is integrally provided with a video reproduction unit 48 to be reproduced and a rotational angular velocity sensor 45 for detecting rotational movement of the head, and binocular parallax is used for the image displayed on the video reproduction unit 48 It is a head-mounted type video and audio 10-05-2019 15 reproduction device 40 which is a stereoscopic video. [0054] The configuration of the audio reproduction unit 42 is the same as that of the audio reproduction unit 10 according to the first embodiment. That is, the audio signal processing unit 5 is configured as shown in FIG. 2, FIG. 8, FIG. 10 or FIG. 12, and the above-mentioned audio signal processing is applied to the input audio signal input through the input terminal 43 to perform D / A conversion. And the sound generator 44L and 44R via an amplifier. [0055] As shown in FIG. 17, the image reproduction unit 48 is an image for the left eye in which binocular parallax occurs, obtained by photographing the same subject 55 with the binocular camera (for the left) 56L and the binocular camera (for the right) 56R. The signal and the video signal for the right eye are input through the input terminals 49L and 49R. The video signal for the left eye is supplied to the video signal processing unit 50L for the left eye, and processed here so as to be fixed in a predetermined direction of the viewing environment according to the detection result of the rotational angular velocity sensor 45. Also, the video signal for the right eye is supplied to the video signal processing unit 50R for the right eye, and is processed here so as to be fixed in a predetermined direction of the viewing environment according to the detection result. That is, in the video signal processing unit 50L and the video signal processing unit 50R, the display positions of the video signal for the left eye and the video signal for the right eye in the video display unit 52L and the video display unit 52R according to the rotation angle of the head. Process to change. When the head rotates to the right, the frame combining position of the video signal for the left eye and the video signal for the right eye moves to the left in the video display unit 52L and the video display unit 52R, and conversely, the head When rotating to the left, the frame combining position moves to the right in the video display unit 52L and the video display unit 52R. For this reason, the video appears to be staying in a certain direction in the viewing environment. Also, the video signal is perceived as a stereoscopic image with a sense of perspective because it captures binocular parallax as described above. [0056] 10-05-2019 16 The video signal processing units 50L and 50R have the same configuration as that shown in FIG. Further, the display areas of the image display units 52L and 52R have the same configuration as that shown in FIG. The description is omitted here. [0057] As described above, the head-mounted video / audio reproduction apparatus 40 according to the second embodiment of the present invention includes the audio signal processing unit and the video signal processing unit 50L according to the rotational movement angle of the head calculated by the microprocessor 46. And 50R to process the sound image localization position and the image position where binocular parallax occurs to stay in a fixed direction outside the head and in the viewing environment, the sound image localization position and the image position overlap as in the prior art. It is possible to eliminate the sense of discomfort due to the sound source, and to always match the positions of the sound source and the actual sound image on the image projected three-dimensionally on the display. Furthermore, the synergetic effect of the stereoscopic image and the sound image makes it possible to obtain clearer sound image localization as compared with the case of only the sound, and to realize realistic stereoscopic video and audio reproduction. [0058] Next, a third embodiment will be described. The third embodiment has a configuration similar to that of the head-mounted video / audio reproduction device 1 shown in FIG. 1, but the video signal processing unit 12 selectively outputs a part of the video signal. Video signal processing. [0059] That is, according to the head rotation angle calculated by the microprocessor 17 from the rotation angular velocity detected by the rotation angular velocity sensor 16, whether the content displayed on the image display units 14L and 14R is fixed in a certain direction of the viewing environment To change. [0060] 10-05-2019 17 The state of selection of such a video area will be described with reference to FIG. After the video signal processing unit 12 outputs the display area 58 of the video signal content entire area 57 to the video display units 14L and 14R in the initial state, the video signal processing unit 12 is turned when the viewer's head is turned to the right. Sequentially output a display area 59, which is an image matching the rotation angle, to the image display units 14L and 15R. Conversely, when the head is turned to the left from this state, it heads in the direction of the display area 58. [0061] Although the video signal processing unit 12 has the same configuration as that shown in FIG. 6 above, although the method of expanding the video data on the image memory 33 is a part of the memory 33 in the first embodiment. On the other hand, the video data is stored in the entire area of the image memory 33, and a part of the actual display area is cut out and displayed. [0062] The configuration and operation of the audio reproduction unit 2 are the same as those of the audio reproduction unit 2 shown in FIG. [0063] As described above, since the video signal processing unit 12 keeps the display content in a certain direction in the viewing environment, the third embodiment enables video and audio reproduction in which the video image and the sound image direction coincide with each other. [0064] Next, a fourth embodiment will be described. In the fourth embodiment, as shown in FIG. 19, the position information of the wearer of the head-mounted unit 65 is input to the video and audio signal generator 61 provided separately from the head-mounted unit 65, and the position information is provided. The head-mounted video / audio reproduction device 60 outputs the video signal and the audio signal according to the video / audio environment so as to be fixed. 10-05-2019 18 [0065] The head rotation angle detected by the rotation angular velocity sensor 66 provided in the head mounting unit 65 is input to the microprocessor 67 and converted into a position information control signal here. The position information control signal is supplied to the video and audio signal generator 61 via the position information output terminal 68. [0066] The video / audio signal generation unit 61 knows the positional information of the wearer of the head-mounted unit 65 according to the positional information control signal, and updates the contents of the video signal and the audio signal. Here, the video and audio signal generation unit 61 synthesizes and outputs the video signal by a method such as computer graphics, stores the audio signal in a large scale memory in advance, and selectively extracts it from the large scale memory. Output. Therefore, the video / audio signal generation unit 61 displays the video signal synthesized by the method such as the computer graphic updated according to the head position of the head mounting unit 65 through the video input terminal 62. It supplies to drive part 63L and 63R. Further, the video / audio signal generation unit 61 reproduces the above-mentioned audio signal selectively extracted from the large-scale memory updated according to the head position of the head-mounted unit 65 wearing user via the audio input terminal 69. It supplies to the part 70. [0067] The sound reproduction unit 70 performs sound signal processing similar to that performed by the sound reproduction unit 1 of FIG. 1 on the sound signal, and a sound generator 71 L disposed to form a headphone in the head mounted unit 65 And supply to 71R. Therefore, the audio signal is processed so that the sound image is fixed at a fixed position outside the head according to the movement of the viewer's head. Further, by using a plurality of audio signals, 10-05-2019 19 three-dimensional sound field reproduction in which a plurality of sound images are localized at an arbitrary position can be performed. [0068] On the other hand, the video display drive units 63L and 63R, to which the video signal is input through the video input terminal 62, are disposed so that the video signal is in the head mounted unit 65 and located near both eyes. It supplies to the image display parts 64L and 64R. The image display units 64L and 64R display the image signal such that the content itself is fixed in the viewing environment. That is, when the head is rotated to the right, the image is such that the image is panned to the right, and conversely, when the head is rotated to the left, the image is panned to the left and moved . For this reason, the display content seems to be staying in a certain direction in the viewing environment. [0069] As described above, the head-mounted video and audio reproduction apparatus 60 according to the fourth embodiment can be configured such that the actual sound image is localized to the sound generation source on the video including the sense of closeness in the video and sound field. Extremely realistic video and audio reproduction becomes possible. [0070] The fifth embodiment will now be described. Similar to the third embodiment, the fifth embodiment has a configuration like the head-mounted type video and audio reproduction device 1 shown in FIG. 1, and the video signal processing unit 12 is an example of a video signal. The video signal processing is performed to selectively output the part, and when the display area reaches the limit of the video signal content, the video and sound image localization positions are synchronized with the head movement and rotated with respect to the viewing environment. [0071] 10-05-2019 20 That is, after the video signal processing unit 12 outputs the display area 58 of the video signal content entire area 57 to the video display units 14L and 14R in the initial state, the video signal processing is performed when the viewer's head is rotated to the right The unit 12 continuously outputs the display area 59, which is an image matching the rotation angle, to the image display units 14L and 15R. Conversely, when the head is turned to the left from this state, it heads in the direction of the display area 58. [0072] Here, as shown in FIG. 20, when the display area 85 reaches the limit of the entire area 57 of the video signal content, the video and sound image localization positions are synchronized with the movement of the head at that time to the viewing environment. When the head is rotated in the reverse direction in this state, both video and sound images are fixed to the viewing environment as long as the video signal content entire area 57 permits. Be done. [0073] As described above, when the viewer usually watches software such as an existing movie by wearing this fifth embodiment, the video and audio need not be fixed within the entire range, for example, as shown in FIG. If the video position is fixed within a predetermined range W of the front F of the viewer M as shown in FIG. 21, the video position and the sound image position will be fixed within a practically sufficient range. The displayed content appears to be in a fixed direction in the viewing environment, and video and audio reproduction in which the video image and the sound image direction coincide can be achieved. In addition, since the time difference and level difference between both ears due to the movement of the head with respect to the audio signal are added as well as listening to the real sound source, the sound image can be localized in the state including the head and the forward direction. [0074] The voice input signal may be supplied to the voice reproduction means on the head-mounted unit wirelessly. In addition, the video input signal may be supplied to the video reproduction means on the head-mounted unit wirelessly. 10-05-2019 21 [0075] According to the present invention, the audio / video reproduction apparatus according to the present invention causes the audio signal processing unit of the audio reproduction means to localize the reproduced sound image outside the head and the sound image direction is a fixed direction of the viewing environment regardless of the rotational movement of the head. Signal processing to fix the image perceptual position in a certain direction of the viewing environment according to the rotational movement of the head according to the rotational motion of the head. The sound image can be localized outside the head, particularly in the forward direction where localization is conventionally considered difficult. In addition, since the image position is fixed at a predetermined position, the sound image localization position and the image position overlap, so that the sense of discomfort due to the mismatch between the both can be eliminated. Furthermore, the synergetic effect of the image and the sound image makes it possible to obtain clearer sound image localization than in the case of only sound, and to realize realistic image and sound reproduction. Furthermore, since sound image out-of-head localization processing can be realized by signal processing with a simple configuration, a low-cost apparatus with excellent portability can be configured in a small size. [0076] In addition, if binocular parallax is added to the video signal to be input, it is possible to reproduce a stereoscopic image with a long distance, and as in the prior art, the sound image localization position and the video position overlap and the discomfort due to the mismatch between both is eliminated. It is possible to always match the positions of the sound source and the actual sound image on the image projected three-dimensionally on the display. Furthermore, the synergetic effect of the stereoscopic image and the sound image makes it possible to obtain clearer sound image localization as compared with the case of only the sound, and to realize realistic stereoscopic video and audio reproduction. [0077] In addition, since the video reproduction means can selectively display a part of the video signal, it is possible to view the sound image localization position and the video position without reducing the amount of video viewed by the viewer for the movement of a certain range of the head. Fixation to the environment can be realized. 10-05-2019 22 [0078] Further, according to the present invention, for example, the present invention is applied to a system capable of synthesizing an image in real time, such as computer graphics, and the contents of the image content itself can be updated and displayed according to the rotational movement of the head. Fixation of the sound image localization position and the image quality to the viewing environment can be realized without reducing the amount of image viewed by the viewer with respect to the movement of the range. [0079] In addition, the sound reproduction means and the video reproduction means fix the sound image direction in a certain direction of the viewing environment only within a certain range centered on the front direction according to the movement of the head. Since the voice direction can be moved according to the movement, the range in which the sound image direction is fixed to a certain direction of the viewing environment is limited even when viewing an image whose display content does not change according to the head movement. Therefore, the direction of the image and the direction of the sound image do not largely deviate, and the forward localization of the sound image by dynamic sound signal processing is enabled, and a wide image display range such as movie viewing is not required. Even with video and audio sources to be watched with, it makes it easy to reproduce video and audio with sufficient reality and realism. [0080] Further, in the present invention, the head mounting unit is provided with the rotation detection means and the sound reproduction means, and the movement of the head is detected, and the image direction and the sound image direction are constant directions of the viewing environment only within a certain range centered on the front direction. Changes as if fixed, and if it exceeds the range, it moves in both the image direction and the sound image direction according to the movement of the head, so that the display content changes according to the movement of the head Even when the information capacity is limited, in the case where the rotational movement range of the head is limited, the image direction and the sound image direction can be fixed in a certain direction of the viewing environment. In addition, it enables the forward localization of the sound image by dynamic sound signal processing, and there is sufficient reality and presence for a system that allows viewing in a state with few normal movements without requiring a wide image display range such as movie viewing. Video and audio reproduction can be easily configured. 10-05-2019 23 [0081] In addition, by incorporating the audio reproduction means in the head-mounted unit, the audiovisual signal having a conventional ordinary audio signal source is directly input without having special recording such as binaural recording, etc. to have localization outside the head It enables immersive voice reproduction and enables a portable virtual reality system to be miniaturized. [0082] Also, by providing the position detection means in the head mounting unit instead of or simultaneously with the rotation angle detection means, not only control when the wearer rotates but also an image and sound image when the position is moved in the viewing environment Control is also possible, and a more realistic virtual reality system can be configured compactly. [0083] Brief description of the drawings [0084] 1 is a block diagram showing a schematic configuration of a first embodiment of a video and audio reproduction apparatus according to the present invention. [0085] 2 is a block diagram showing a detailed configuration of the audio signal processing unit used in the first embodiment. [0086] 3 is a schematic view showing a listening environment assumed by the audio signal processing unit used in the first embodiment. [0087] 4 is a characteristic diagram of the delay time added by the time difference adding circuit of the audio signal processing unit used in the first embodiment. [0088] 10-05-2019 24 5 is a characteristic diagram of the level difference added by the level difference adding circuit of the audio signal processing unit used in the first embodiment. [0089] 6 is a block diagram showing a detailed configuration of a video signal processing unit used in the first embodiment. [0090] 7 is a schematic view of a display area of the video reproduction unit used in the first embodiment. [0091] FIG. 8 is a block diagram in the case where the audio signal processing unit uses a time difference adding circuit and a frequency characteristic control circuit. [0092] FIG. 9 is a characteristic diagram of frequency control performed by the frequency characteristic control circuit. [0093] FIG. 10 is a block diagram when the audio signal processing unit uses a phase difference adding circuit and a level difference adding circuit. [0094] 11 is a change characteristic diagram of the phase difference added by the phase difference adding circuit. [0095] 12 is a block diagram in the case where the audio signal processing unit uses a phase difference adding circuit and a frequency characteristic control circuit. [0096] 13 is a schematic block diagram of a pressure-transfer vibration gyro device applicable to the 10-05-2019 25 rotational angular velocity sensor used in the first embodiment. [0097] 14 is a schematic configuration diagram of an analog angle detector that can be applied to the rotational angular velocity sensor used in the first embodiment. [0098] FIG. 15 is a schematic block diagram of a digital angle detector applicable to the rotational angular velocity sensor used in the first embodiment. [0099] 16 is a block diagram showing a schematic configuration of a second embodiment of the video and audio reproduction apparatus according to the present invention. [0100] FIG. 17 is a schematic view for explaining a video signal in which binocular parallax occurs which is input in the second embodiment. [0101] 18 is a schematic diagram for explaining the operation of the video reproduction unit of the third embodiment. [0102] 19 is a schematic diagram for explaining a fourth embodiment of the video and audio reproduction apparatus according to the present invention. [0103] FIG. 20 is a schematic view for explaining the operation of the video reproduction unit of the fifth embodiment of the video and audio reproduction device according to the present invention. [0104] 21 is a schematic diagram for explaining the operation of the fifth embodiment. 10-05-2019 26 [0105] FIG. 22 is a schematic view of a conventional head-mounted type video and audio reproduction apparatus. [0106] 23 is a block diagram of a conventional video and audio reproduction apparatus. [0107] Explanation of sign [0108] Reference Signs List 1 head-mounted type video and audio reproduction device 2 audio reproduction unit 5 audio signal processing unit 10 video reproduction unit 12 video signal processing unit 14 video display unit 15 head mounting unit 16 rotational angular velocity sensor 17 microprocessor 10-05-2019 27
© Copyright 2021 DropDoc