JP2015071320

Patent Translate
Powered by EPO and Google
Notice
This translation is machine-generated. It cannot be guaranteed that it is intelligible, accurate,
complete, reliable or fit for specific purposes. Critical decisions, such as commercially relevant or
financial decisions, should not be based on machine-translation output.
DESCRIPTION JP2015071320
PROBLEM TO BE SOLVED: To provide "a conversation support device, a conversation support
method and a conversation support program" capable of outputting a voice by a driver toward a
rear seat. SOLUTION: A conversation support program 140 according to the present embodiment
comprises an image data acquisition unit 142 for acquiring image data for monitoring the
driver's gaze etc., a voice data acquisition unit 144 for acquiring driver's voice data, Based on the
image data acquired by the audio processing unit 146 that performs processing such as cutting
noise of audio data, the audio data output unit 148 that outputs the audio data to the rear seat,
and the driver of the rear seat based on the image data acquired by the image data acquisition
unit 142 When it is determined that there is a conversation operation, it is determined that the
driver's voice data is output to the back seat, and it is determined that there is no conversation
operation. The output control unit 152 controls the audio data output unit 148 so that the
driver's voice data is not output to the rear seat when the driver's voice data is output. [Selected
figure] Figure 4
Conversation support device, conversation support method and conversation support program
[0001]
The present invention relates to a conversation support apparatus having a function of
supporting a passenger's conversation in an in-vehicle environment, and more particularly to an
on-vehicle apparatus having a conversation support function of outputting a voice of a front seat
passenger to a rear seat.
[0002]
09-05-2019
1
In the in-vehicle environment, when passengers are talking, it may be difficult to hear the other
party's voice due to the relationship between road noise and the riding position.
Therefore, the in-vehicle apparatus includes a conversation support function that supports
mutual conversation by means of in-vehicle microphones and speakers disposed at a plurality of
positions such as each seat. The conversation support function is, for example, for acquiring the
voice of the front passenger by an in-car microphone, adjusting the volume of the voice, and the
like, and outputting it to the rear seat. As a result, the user can hear the voice that is scraped off
by road noise and the voice of the front seat passenger that is difficult for the rear seat passenger
to hear, and the passenger can talk without any problem in the in-vehicle environment. You can
enjoy
[0003]
Patent Document 1 relates to a vehicle interior sound field control system capable of outputting a
passenger's voice to a speaker of the other party by selecting the other party of the conversation
so that conversation in the car can be appropriately performed. Is disclosed. Patent Document 2
is a seat occupant information presentation system that can control the voice of each seat
occupant output from the speaker so that the sum of the direct volume and the speaker volume
is the same for all the occupants by the microphone volume and the seat position. Discloses
technology related to Patent Document 3 discloses a technique relating to a vehicle conversation
support device that can communicate without problems between occupants seated in a plurality
of seats disposed in a vehicle cabin.
[0004]
JP, 2005-161873, A JP, 11-355, 748, JP, 11-342,799, A
[0005]
As described above, the on-vehicle device mounted on the vehicle is provided with a conversation
support function to support conversation between passengers, but in the conventional
conversation support function, the driver and the passenger on the passenger seat Even in the
case of the conversation between the two, the voice may be output to the rear seat, which may be
annoying for the rear seat passenger.
09-05-2019
2
In particular, when the passengers in the rear seat have a conversation, if the conversation
between the passengers in the front seat is output from the speaker, it becomes noise and feels
uncomfortable. On the other hand, there is also a conversation function that can select the other
party to talk with, but this requires an instruction by the user operation, so while driving, the
driver's attention is scattered and it interferes with the driving operation I will.
[0006]
The present invention solves such conventional problems, and provides a speech support device,
a speech support method, and a speech support program capable of appropriately outputting a
voice by a front seat passenger to a rear seat passenger. The purpose is
[0007]
A conversation support device according to the present invention outputs a voice input by the
voice input unit, a detection unit that detects the line of sight of the passenger in the front seat or
the direction of the face, a voice input unit that inputs voice, and A voice module having a voice
output unit; determination means for determining whether or not there is a conversation
operation from a front seat passenger to a rear seat passenger based on the detection result of
the detection means; And control means for controlling transfer of voice data among the plurality
of voice modules arranged, and when it is determined by the determination means that there is a
conversation operation, the control means is configured to receive from the voice module of the
front seat The voice communication between the passengers by transferring the voice data to the
voice module of the back seat and not transferring the voice data from the voice module of the
front seat to the voice module of the back seat when it is determined that there is no
conversation operation. It is possible to assist.
[0008]
Preferably, the conversation support device further includes determination means for
determining a seat where a passenger is present, and the control means transfers voice data to
the voice module of the seat determined by the determination means.
[0009]
Preferably, the detection means includes an imaging camera for imaging a driver's face, and the
determination means determines the presence or absence of the conversation operation based on
09-05-2019
3
the image data from the imaging camera.
The determination means determines that there is conversation operation when the driver's gaze
is directed to the rearview mirror.
[0010]
The conversation support method according to the present invention comprises the steps of:
detecting a line of sight of a passenger in a front seat or the direction of a face; inputting voice by
using a voice module; inputting voice by the voice module; A voice output step of outputting the
input voice; a determination step of determining whether or not there is a conversation operation
from a front seat passenger to a rear seat passenger based on the detection result of the
detecting step; Control step of controlling transfer of voice data between a plurality of voice
modules disposed in the seat, and when it is determined in the determination step that there is
conversation operation, the control step comprises: voice module of the front seat Audio data
from the front seat to the voice module in the back seat, and when it is determined that there is
no conversation operation, the voice data from the voice module in the front seat is By not
transfer the data of the rear seat to the sound module, it is possible to support a conversation
between the passenger.
[0011]
A conversation support program according to the present invention is executed in an on-vehicle
apparatus, and includes a detection step of detecting a line of sight or a face direction of a
passenger of a front seat, a voice input step of inputting voice by a voice module, and the voice
module. Determines whether or not there is a conversation operation from the passenger of the
front seat to the passenger of the rear seat based on the detection result of the voice detection
step of outputting the voice input in the voice input step and the detection step. And a control
step of controlling transfer of voice data among a plurality of voice modules disposed in each
seat, and when it is determined in the determination step that there is a conversation operation,
the control step includes: The voice data from the front seat voice module is transferred to the
back seat voice module, and when it is determined that there is no conversation operation, the
front By not transfer audio data from the audio module of the seat of the rear seat to the sound
module, it is possible to support a conversation between the passenger.
[0012]
According to the present invention, it is possible to determine the presence or absence of
09-05-2019
4
conversation operation between the front seat and the rear seat by detecting the direction of the
line of sight and the face by the front seat passenger.
Then, when it is determined that there is a conversation operation, the transfer of voice data is
controlled so that the voice by the passenger of the front seat is output to the back seat, so that
between the passengers of the front seat and the rear seat The conversation support is executed
only in the situation where the conversation is performed, and in the other situations, it is
possible to avoid the negative effects of the conversation support function.
[0013]
It is a figure which shows the arrangement configuration of each device in the vehicle-mounted
apparatus which concerns on the Example of this invention.
It is a figure which shows the connection relation of the main body part of a vehicle-mounted
apparatus, and each device.
It is a block diagram showing an example of 1 composition of an in-vehicle device concerning an
example of the present invention.
It is a block diagram showing the functional composition of the conversation support program
concerning the example of the present invention. It is a flowchart explaining the conversation
assistance operation ¦ movement of the vehicle-mounted apparatus which concerns on a present
Example.
[0014]
Next, embodiments of the present invention will be described in detail with reference to the
drawings. The conversation support device according to the present invention is implemented in
an on-vehicle device in a preferred embodiment. The in-vehicle apparatus utilizes a conversation
support function that supports conversation between passengers in the in-vehicle environment,
and detects the line of sight of the front seat passenger or the direction of the face to allow the
front seat passenger to rearward. It is possible to automatically determine the conversation
09-05-2019
5
operation to the seat occupants, and in the case of a conversation between front seat occupants,
it is possible to control so that the voice is not output to the rear seat.
[0015]
The on-vehicle apparatus is generally an electronic apparatus mounted on a vehicle, and can
have various functions in addition to the conversation support function. For example, the invehicle device has a function of reproducing audio data and video data, a navigation function of
guiding a route to a destination, a function of preventing a driver from falling asleep, a function
of receiving television / radio broadcasting, and a network such as the Internet Can be equipped
with communication functions and the like. Furthermore, the in-vehicle apparatus can connect
various devices such as in-vehicle cameras, in-car microphones for hands-free, in-vehicle
speakers, etc. to the main body, and in a preferred embodiment, the main body and the external
device are Bluetooth etc. Connected by wireless communication. The hands-free function enables
voice input / output to the in-vehicle apparatus, and in this case, voice input / output is
performed via an in-vehicle microphone or an in-vehicle speaker.
[0016]
In a preferred embodiment of the in-vehicle apparatus, the in-vehicle camera is disposed at a
position facing the rearview mirror and each passenger, and detects the line of sight to the
rearview mirror and the face orientation when talking with the rear seat passenger can do.
Further, the voice module is arranged to correspond to each seat of the vehicle, and the in-car
microphone acquires the voice of the passenger sitting on the seat by the hands-free function. In
addition, the in-vehicle speaker can output, for example, audio data to the rear seat so that the
passenger who is the conversation partner can hear it. Devices such as an in-vehicle camera, an
in-vehicle microphone, and an in-vehicle speaker may be connected to the main unit by wire as
well as wireless.
[0017]
In addition, the in-vehicle device has a voice processing function, removes road noise from voice
data acquired through an in-vehicle microphone, performs voice recognition processing, and
extracts only voice data relating to a passenger's speech. Can.
[0018]
09-05-2019
6
Next, an outline of the conversation support function by the on-vehicle apparatus according to
the embodiment of the present invention will be described.
For example, when the driver talks with the passenger in the rear seat, there is a problem that it
is difficult for the passenger in the rear seat to hear the language spoken by the driver because of
the relationship between the riding positions. In such a scene, the conversation support function
of the in-vehicle apparatus can acquire the voice emitted by the driver with the in-vehicle
microphone and can output the voice from the speaker in the rear seat.
[0019]
However, if the conversation between the driver and the passenger in the front passenger seat is
output in the direction of the rear seat, the person in the rear seat may feel rather annoying.
Therefore, in the on-vehicle apparatus according to the present invention, when the driver
detects a gesture (speech operation) to talk to a passenger in the rear seat, the support of the
conversation is performed.
[0020]
The in-vehicle apparatus detects, in a preferred embodiment, for example, the gaze of the driver
looking at the rear seat passenger over the rearview mirror, or the driver tilts the face slightly
backward so as to talk to the rear seat passenger. It is determined that a proper face direction is
detected, and it is determined that a conversation between the driver and the rear seat between
the passengers is established, and the driver's voice is output toward the rear seat.
[0021]
FIG. 1 is a view showing an arrangement configuration of each device in the in-vehicle apparatus
of the present embodiment.
The on-vehicle apparatus 10 according to the present embodiment is an electronic apparatus
mounted inside the vehicle 20 as shown in the figure, and is connected to the main body 40 at
the center front and the main body 40, and the driver And a plurality of audio modules 80a to
09-05-2019
7
80d arranged to correspond to the respective seats A to D. In the preferred embodiment, each
seat is fitted with a weight sensor, and the output of the weight sensor is provided to the body
40. Then, in the main body 40, the seat in which the passenger is present is recognized.
[0022]
FIG. 2 is a diagram showing a connection relationship between the main body 40 and each
device. The main body unit 40 includes the touch panel 42 on the front surface thereof, and can
display various screens such as a menu and a peripheral map, and can perform various settings
in response to user input by touch operation. An on-vehicle camera 60 attached to the rearview
mirror 62 and the like and audio modules 80 a to 80 d arranged to correspond to the respective
seats A to D are connected to the main body 40. The voice modules 80a to 80d respectively
include in-car microphones 82a to 82d and in-vehicle speakers 84a to 84d, and each module is
connected to the main body 40 by wireless communication or the like as described later. Further,
each module is attached with an operation unit for adjusting the volume from the speaker.
[0023]
Preferably, Bluetooth (registered trademark) BT HSP (Headset Profile) is used for the connection
between the main body 40 and the audio modules 80a to 80d. The BT HSP is a profile that
enables transmission and reception of audio data between connected devices, and the profile is
supported by the main unit 40 and each of the audio modules 80a to 80d. It controls the transfer
of voice data between modules 80. That is, the main body unit 40 performs transfer control on
which speaker the audio data input from the microphones 82a to 82d is to be output. In a
preferred embodiment, the main body 40 automatically determines the direction of the driver's
conversation from the driver's operation, and when it is determined that the conversation with
the rear seat passenger is made, the driver's voice data Is output from the rear seat speaker, and
if it is determined not to be such a conversation, the driver's voice data is not output from the
rear seat speaker.
[0024]
FIG. 3 is a block diagram showing an example of the configuration of the in-vehicle apparatus 10.
As shown in FIG. The in-vehicle device 10 monitors an operation of a driver, an in-vehicle camera
60, an audio module 80 (80a to 80d) for acquiring a voice of a passenger or outputting the voice
09-05-2019
8
toward a rear seat, CD, DVD, Blu-ray disc, It plays audio data and video data stored in hard disk
devices, etc., multimedia playback unit 100 that receives digital television broadcasts, AM / FM
radio broadcasts, etc., displays a road map around the position of the vehicle, and a destination A
navigation unit 102 for guiding a guidance route up to, a communication unit 104 for enabling
data communication with an external network, a storage unit 106 for storing programs, data, etc.,
an output unit 108 for outputting voice, images, etc. , User operation, input unit 110 for
inputting voice, etc., microcontrollers and Configured to include a control unit 120 for
controlling each unit by executing a program includes a processor or the like. The control unit
120 also receives the detection result of the weight sensor (passenger sensor) as described
above, and identifies the seat where the passenger is present based on the detection result. In
addition, the structure shown here is an illustration, and the vehicle-mounted apparatus 10 is not
limited to such a structure.
[0025]
In order to monitor the driver's gaze direction and face direction, for example, the on-vehicle
camera 60 is attached to a rear view mirror 62 at the upper center of the front as shown in FIG.
1 and captures an image of the driver's face. . In this embodiment, the driver sitting on the front
seat A or B uses the on-vehicle camera 60 to detect the front passenger's gesture (conversational
motion) when talking to the passengers sitting on the back seats C and D. However, other sensors
may be used as long as they can detect the line of sight, the direction of the face, and the like.
Furthermore, as long as the line of sight or the direction of the face can be detected, the
mounting position of the on-vehicle camera 60 is not necessarily limited to the rearview mirror
62, and may be mounted on a windshield, an instrument panel including meters and instruments,
a main body 42, etc. Good. Furthermore, a plurality of in-vehicle cameras 60 may be prepared
according to at least the number of seats. On the other hand, the acquired image data is
subjected to image analysis by the control unit 120 in order to detect the sight line direction, the
face direction, and the like. The image data is also used for the driver's sleep prevention function
in addition to the conversation support function.
[0026]
The audio module 80 includes an in-car microphone 82 for acquiring a passenger's voice and an
on-vehicle speaker 84 for outputting audio data acquired from the in-car microphone 82, and
each of the audio modules 80a to 80d is shown in FIG. , And each of the seats A to D is wirelessly
connected to the control unit 120. The control unit 120 identifies the devices of the audio
modules 80a to 80d in connection by the BT HSP, and controls transmission and reception of
09-05-2019
9
audio data between the audio modules. In a preferred embodiment, the control unit 120
automatically determines the direction of the conversation from the driver to the passenger of
the rear seat based on the conversation operation of the driver, and the voice data acquired by
the in-car microphone 82a of the driver is The sound is output from the rear seat speaker 84c or
84d on which the person is sitting.
[0027]
The communication unit 104 includes, for example, a Bluetooth module (hereinafter, abbreviated
as a BT module) as a connection means for enabling input and output of audio data between the
audio modules 80. The BT module includes, for example, a profile such as HSP, and enables
transmission and reception of audio data between the audio module 80 and the control unit 120.
[0028]
The control unit 120 controls the functions of the respective units in accordance with the various
programs read from the storage unit 106. Here, processing of audio data when the control unit
120 executes a conversation support program, which is one of them, will be described.
[0029]
FIG. 4 is a diagram showing a functional configuration of the conversation support program 140.
As shown in FIG. As shown in the figure, the conversation support program 140 is input from the
in-vehicle microphone 82 and an image data acquisition unit 142 for acquiring image data from
the on-vehicle camera 60 for monitoring the direction of the driver's eyes or face. A voice data
acquisition unit 144 for obtaining voice data, a voice processing unit and 146 for cutting noise
from the obtained voice data and extracting voice data according to a passenger's speech, and
voice data subjected to voice processing On the basis of the audio data output unit 148 that
causes the selected vehicle-mounted speaker 84 to output and the image data acquired by the
image data acquisition unit 142, it is determined whether there is a conversation operation in
which the driver talks to the passenger in the rear seat Control the audio data output unit 148
based on the determination results of the conversation operation determination unit 150 and the
conversation operation determination unit 150. And an output control unit 152.
09-05-2019
10
[0030]
The image data acquisition unit 142 acquires image data of the driver captured by the on-vehicle
camera 60 and provides the conversation operation determination unit 150 with the image data.
In a preferred embodiment, the image data acquisition unit 142 may trim the image data of the
driver's face or the line of sight from the captured image data and provide it to the conversation
operation determination unit 150. . In this case, the trimming range is set in advance.
[0031]
The voice processing unit 146 has a filter function of cutting road noise such as vehicle engine
noise, friction noise between road and tire, wind noise due to traveling, and can remove road
noise from acquired voice data. In addition, the voice processing unit 146 has a voice recognition
function, and can extract only voice data related to a passenger's speech from the obtained voice
data. The audio data output unit 150 can output the audio data subjected to the audio processing
by the audio processing unit 146 from the selected audio module 80.
[0032]
The conversation operation determination unit 150 can analyze the image data of the driver
acquired by the image data acquisition unit 142 and determine whether the driver has performed
an operation to talk to a passenger in the rear seat. For example, when a driver talks to a
passenger in the rear seat, he tends to look at the other person over the rearview mirror.
Alternatively, the driver tends to turn his face towards the rear seat when speaking to the rear
seat passenger. The conversation operation determination unit 150 determines the direction of
conversation from the front seat to the rear seat based on such a driver's gesture (a conversation
operation).
[0033]
The determination condition of the presence or absence of the conversation by the driver may
include the presence or absence of the utterance by the driver. In this case, the conversation
operation determination unit 150 assumes that voice data is input from the voice module 80 of
09-05-2019
11
the driver's seat as a precondition, and if such voice data is not input, the driver's gaze Even if
there is a change in the face, it may be determined that there is no conversation from the driver
to the rear seat passenger.
[0034]
Furthermore, in addition to changes in the driver's line of sight and face, a sign (action) of
conversation start may be determined in advance, and the sign may be used as a determination
condition. For example, the driver may perform a wink multiple times, the driver may open the
mouth, or the driver may move the eyebrows.
[0035]
The output control unit 152 controls the output of the audio data output unit 148 according to
the determination as to whether or not there is a conversation operation by the conversation
operation determination unit 150. When it is determined that the driver talks the rear seat to the
passenger, the audio data output unit 148 outputs the audio data input from the in-car
microphone 82a of the audio module 80a corresponding to the driver's seat to the rear seat The
voice module 80c or 80d corresponding to the passenger of the above is output from the speaker
84c or 84d.
[0036]
Next, the conversation support operation of the on-vehicle apparatus according to the
embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a
flowchart for explaining the conversation support operation by the on-vehicle apparatus.
[0037]
When the power of the in-vehicle apparatus is turned on, the conversation support function is
activated (S200). When the conversation support function is activated, the control unit 120 reads
the conversation support program 140 from the storage unit 106, and controls each unit
according to the conversation support program 140. It can be set by the user whether or not to
09-05-2019
12
enable the conversation support function, and is activated simultaneously with the power-on of
the in-vehicle device 10 if it is set in advance as valid. Also, after the power of the in-vehicle
apparatus is turned on, the conversation support function may be activated according to the
instruction of the user.
[0038]
When the conversation support function is activated, the image data acquisition unit 142
acquires image data output from the in-vehicle camera 60 (S201). The image data is for
monitoring the driver's motion, and includes an image of the driver's face. The conversation
operation determination unit 150 analyzes the acquired image data, recognizes the driver's gaze
or face from the image data, and detects the driver's gaze or face direction (S202).
[0039]
Next, the conversation operation determination unit 150 determines whether or not the
conversation operation is to talk to the passenger in the rear seat according to the change in the
direction of the driver's eyes or face (S203). For example, when the line of sight of the driver is
changed by a certain angle or more in the direction of confirming the occupant of the rear seat
through the rearview mirror, the conversation operation determination unit 150 makes a face
orientation a certain angle so as to talk to the rear seat. When it is changed, it is determined that
the conversation operation to the rear seat passenger has occurred.
[0040]
The conversation operation determination unit 150 determines that a voice is input in the
driver's seat voice module or a voice is generated in the passenger's back seat voice module as a
determination condition of occurrence of the conversation operation to the passenger in the rear
seat. The input may be a weighting requirement.
[0041]
When it is determined by the conversation operation determination unit 150 that there is no
conversation operation, the output control unit 152 may be configured to acquire voice data
from the voice module of the driver's seat by the voice data acquisition unit 144. The audio data
output unit 148 is controlled so that the audio data is not output from the audio module of the
back seat (S204).
09-05-2019
13
[0042]
For example, the voice data output unit 148 stops transfer of voice data from the driver's voice
module.
Alternatively, the output control unit 152 may output a control signal such that the speaker of
the audio module of the rear seat in which the passenger is present is muted.
As a result, the rear seat passenger does not feel much discomfort because it does not receive
support for conversations unrelated to him.
[0043]
On the other hand, when it is determined by the conversation operation determination unit 150
that there is a conversation operation, after the driver's voice data acquired by the voice data
acquisition unit 144 is processed by the voice processing unit 146, the voice data is The output
control unit 152 transfers the voice to the voice module corresponding to the rear seat of the
passenger, and the voice is output from the voice module (S205). This allows the passenger in
the rear seat to hear the voice of the conversation-assisted driver.
[0044]
As described above, according to the present embodiment, by detecting the line of sight and the
direction of the face of the driver, it is automatically determined whether or not the conversation
operation between the driver and the passenger of the rear seat is present, and the conversation
operation is present. When it is determined, the driver's voice is output from the rear seat, so that
the rear seat passenger can receive support for conversations related to him. In addition, the
driver can drive safely because it is not necessary to indicate that it is a conversation with a rear
seat passenger.
[0045]
09-05-2019
14
In the above embodiment, an example in which the driver assists the conversation to the rear
seat passenger has been described. However, the present invention is not limited to the driver
but also to the front seat passenger, for example, the passenger front passenger. It can apply.
Furthermore, although the example which mounted the vehicle-mounted apparatus in the fourseater vehicle was demonstrated in the said Example, you may mount not only in this but in the
bus ¦ bath, aircraft, etc. in which several seats are arranged back and forth. In this case, the front
seat is not limited to the seat in the front row, and may be a seat relatively forward with respect
to the rear seat.
[0046]
Furthermore, in the above embodiment, the presence or absence of the passenger is determined
by the weight sensor. However, the present invention is not limited to this, based on the image
data from the imaging camera for imaging the inside of the car and the output signal of the
seatbelt wearing sensor. The presence or absence may be determined. Furthermore, the boarding
position data of the passenger can be created, and the output control unit can perform
conversation support based on the boarding position data.
[0047]
The present invention is not limited to the embodiments described above, and various
modifications and changes are possible within the scope of the subject matter of the invention
described in the claims.
[0048]
10: in-vehicle device 20: vehicle 40: main body 42: touch panel 60: in-vehicle camera 62: room
mirror 80 (a to d): audio module 82 (a to d): in-vehicle microphone 84 (a to d): in-vehicle speaker
100: multimedia reproduction unit 102: navigation unit 104: communication unit 106: storage
unit 108: output unit 110: input unit 120: control unit 140: conversation support program 142:
image data acquisition unit 144: audio data acquisition unit 146: audio Processing unit 148:
voice data output unit 150: conversation operation determination unit 152: output control unit
09-05-2019
15