EP3598259B1

Movatterモバイル変換

Info

Publication number: EP3598259B1
Application number: EP19184176.6A
Authority: EP
Inventors: Motoshi ANABUKI
Original assignee: Panasonic Intellectual Property Management Co Ltd
Current assignee: Panasonic Intellectual Property Management Co Ltd
Priority date: 2018-07-19
Filing date: 2019-07-03
Publication date: 2021-09-01
Anticipated expiration: 2039-07-03
Also published as: US11450153B2; CN116343483A; US20200027284A1; CN110738843B; JP7576781B2; US11145145B2; EP3598259A1; CN110738843A; US20210407223A1; JP2023106448A

Description

FIELD

The present disclosure relates to an information processing method and an information processing system.

BACKGROUND

In recent years, self-driving techniques have been introduced and increasingly researched and developed. It is expected that, along with the advancement of the self-driving techniques, unmanned self-driving vehicles will become widely used in the future. From a standpoint of safety, the operation of unmanned self-driving vehicles requires monitoring of the vehicles by operators in remote places. Some techniques for monitoring vehicles have been proposed.
For example, Patent Literature (PTL) 1 discloses a technique for determining a remote control device that sends a remote control request to a control center when a vehicle enters a remote control target area, and allows the control center to remotely control the vehicle for which the remote control request has been sent.
Moreover,PTL 2 discloses a technique for changing the departure times of trains according to the number of passengers entering a station.

CITATION LIST

PATENT LITERATURE

PTL 1:Japanese Unexamined Patent Application Publication No. 2017-147626
PTL 2:Japanese Unexamined Patent Application Publication No. 2015-182623

WO 2018/037900A1 discloses a driving support device, a method, a mobile body, and a program, whereby appropriate driving support can be provided. The driving support device comprises a control unit that performs driving support processing for the host vehicle, on the basis of peripheral vehicle information relating to peripheral vehicles present in an area corresponding to the number of occupants in the host vehicle.
US 2018/137595A1 discloses a display device mounted inside a vehicle. The display device includes a first communication unit configured to be connected to a camera and receive an image of an outside of the vehicle captured by the camera; a location information acquiring unit configured to acquire location information of the vehicle; a control unit configured to determine a destination of the vehicle and control a display time point of the image received through the first communication unit on the basis of the destination of the vehicle and the acquired location information; and a display unit configured to display the image captured by the camera according to a control signal of the control unit.
US 2017/277191A1 discloses arranging a pickup between a driverless vehicle and a passenger. For instance, dispatch instructions dispatching the vehicle to a predetermined pickup area in order to pick up the passenger are received by the vehicle which begins maneuvering to the predetermined pickup area. While doing so, the vehicle receives from the passenger's client computing device the device's location. An indication that the passenger is interested in a fly-by pickup is identified. The fly-by pickup allows the passenger to safely enter the vehicle at a location outside of the predetermined pickup area and prior to the one or more processors have maneuvered the vehicle to the predetermined pickup area. The vehicle determines that the fly-by pickup is appropriate based on at least the location of the client computing device and the indication, and based on the determination, maneuvers itself in order to attempt the fly-by pickup.

SUMMARY

TECHNICAL PROBLEMS

In the conventional technique disclosed inPTL 1, when vehicles enter a remote control target area, an operator simultaneously considers the vehicles as remote control targets. As a result, a single operator finds it difficult to remotely control the vehicles at the same time.
Moreover, the conventional technique disclosed inPTL 2 is intended for an operator in a single vehicle, and does not assume that a single operator manages vehicles at the same time.
In view of the above, the present disclosure provides an information processing method, an information processing apparatus, and an information processing system that make it possible to reduce the burden of an operator who monitors or controls vehicles while assuring safety for the vehicles by the monitoring or controlling.

SOLUTION TO PROBLEMS

In order to achieve the above object, an information processing method according toclaim 1 is provided.
It should be noted that these general and specific aspects may be implemented using a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or recording media.
Aspects and embodiments of the present disclosure are described below with reference to the accompanying drawings. Aspects and embodiments described below which do not fall within the scope of the claims are to be interpreted as examples useful for understanding the invention.

ADVANTAGEOUS EFFECT

The information processing method etc. of the present disclosure make it possible to reduce the burden of an operator who monitors or controls vehicles while assuring safety for the vehicles by the monitoring or controlling.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an information processing system according toEmbodiment 1.
FIG. 2A is a table illustrating priority level information according toEmbodiment 1.
FIG. 2B is a table illustrating a database according toEmbodiment 1.
FIG. 2C is a table illustrating a database according toEmbodiment 1.
FIG. 3 is a sequence diagram illustrating the operation of the information processing system according toEmbodiment 1.
FIG. 4A is a diagram illustrating a presentation made by a terminal according toEmbodiment 1.
FIG. 4B is a diagram illustrating a presentation made by the terminal according toEmbodiment 1.
FIG. 4C is a diagram illustrating a presentation made by the terminal according toEmbodiment 1.
FIG. 4D is a diagram illustrating a presentation made by the terminal according toEmbodiment 1.
FIG. 4E is a diagram illustrating a presentation made by the terminal according toEmbodiment 1.
FIG. 5 is a flow chart illustrating determination of a priority level by the operation of an information processing apparatus according toEmbodiment 1.
FIG. 6 is a flow chart illustrating calculation of a boarding and alighting completion time using a database by the operation of the information processing apparatus according toEmbodiment 1.
FIG. 7 is a block diagram of an information processing system according toEmbodiment 2.
FIG. 8A is a table illustrating a database according toEmbodiment 2.
FIG. 8B is a table illustrating a database according toEmbodiment 2.
FIG. 8C is a table illustrating a database according toEmbodiment 2.
FIG. 9 is a sequence diagram illustrating the operation of the information processing system according toEmbodiment 2.
FIG. 10A is a diagram illustrating a presentation made by a terminal according toEmbodiment 2.
FIG. 10B is a diagram illustrating a presentation made by the terminal according toEmbodiment 2.
FIG. 10C is a diagram illustrating a presentation made by the terminal according toEmbodiment 2.
FIG. 10D is a diagram illustrating a presentation made by the terminal according toEmbodiment 2.
FIG. 10E is a diagram illustrating a presentation made by the terminal according toEmbodiment 2.
FIG. 10F is a diagram illustrating a presentation made by the terminal according toEmbodiment 2.
FIG. 11 is a flow chart illustrating determination of a priority level by the operation of an information processing apparatus according toEmbodiment 2.

DESCRIPTION OF EMBODIMENTS

An information processing method according to one aspect of the present disclosure includes: obtaining, from vehicles, sensor data obtained from sensors of the vehicles; performing object detection processes each using a different one of the sensor data; determining a style of a presentation for monitoring or controlling the vehicles by a remote control operator that is an operator in a remote place, using results of the object detection processes; and controlling the presentation made to the remote control operator by a presentation device according to the style
Since a presentation for monitoring or controlling (hereinafter collectively and simply also referred to as monitoring) vehicles is made in a style determined using the results of objection detection processes, an operator who monitors vehicles can easily find out vehicles to be monitored. In other words, it is possible to reduce the number of vehicles to be simultaneously monitored by the operator while diminishing the operator's risk for overlooking vehicles to be monitored. Accordingly, it is possible to reduce the burden of the operator who monitors vehicles while assuring safety for vehicles by the monitoring.
An information processing apparatus according to one aspect of the present disclosure includes: an obtaining unit that obtains, from vehicles, sensor data obtained from sensors of the vehicles; a detection processor that performs object detection processes each using a different one of the sensor data; a determination unit that determines a style of a presentation for monitoring or controlling the vehicles by a remote control operator that is an operator in a remote place, using results of the object detection processes; and a presentation controller that controls the presentation made to the remote control operator by a presentation device according to the style determined.
An information processing system according to one aspect of the present disclosure includes the information processing apparatus and the presentation device.
The information processing apparatus and the information processing system produce the same advantageous effects as above.
The information processing method according to one aspect of the present disclosure further includes: determining priority levels of the vehicles based on the results of the object detection processes; and determining the style of the presentation according to the priority levels.
Since a presentation device makes a presentation for monitoring or controlling vehicles according to priority levels of the vehicles, an operator who monitors the vehicles can find out vehicles to be preferentially monitored among the vehicles. For this reason, the operator can monitor any vehicle having a higher priority level than those of other vehicles, and thus it is possible to reduce the burden of the operator while assuring safety for vehicles.
Moreover, the information processing method according to one aspect of the present disclosure further includes: obtaining states of the vehicles from the vehicles; and determining the priority levels based on the results of the object detection processes and the states of the vehicles.
It is possible to determine priority levels of vehicles more appropriately using not only the results of object detection processes but also the states of the vehicles. This allows an operator who monitors vehicles to further easily find out vehicles to be preferentially monitored among the vehicles. For this reason, the operator can further easily monitor any vehicle having a higher priority level than those of other vehicles, and thus it is possible to further reduce the burden of the operator while assuring safety for vehicles.
Moreover, in the information processing method according to one aspect of the present disclosure, in each of the object detection processes, at least one of a boarding passenger and an alighting passenger is detected for a stopped vehicle, and the style of the presentation is determined according to the at least one of the boarding passenger and the alighting passenger detected.
It is conventionally difficult to control timing for departure by determining completion of boarding a vehicle or alighting from a vehicle. For example, it is difficult to determine whether a person close to a vehicle is a person planning to board, based on sensor data. When an erroneous determination is made, a boarding and alighting passenger may be put in danger. On the other hand, although it is conceivable that a monitoring operator controls timing for departure, constant monitoring of boarding and alighting states of vehicles increases the burden of the operator. In view of the above, with the present configuration, since a style of a presentation for monitoring is controlled according to boarding and alighting passengers, an operator can easily find out vehicles to be monitored. Accordingly, it is possible to reduce the burden of the operator while assuring safety for vehicles at the time of boarding and alighting. It should be noted that the present configuration is effective even when a departure is automatically controlled. This is because the departure may cause danger as described above, and the operator is required to preferentially perform monitoring.
Moreover, in the information processing method according to one aspect of the present disclosure, in each of the object detection processes, an attribute or a count of at least one of the boarding passenger and the alighting passenger is detected, and the style of the presentation is determined according to the attribute or the count detected.
Since a presentation is made in a style according to elements regarding priority levels for monitoring at the time of boarding and alighting, an operator can more accurately find out vehicles to be monitored.
Moreover, the information processing method according to one aspect of the present disclosure further includes: calculating a time for completion of at least one of the boarding passenger boarding the stopped vehicle and the alighting passenger alighting from the stopped vehicle, using the attribute or the count; and determining the style of the presentation according to the time calculated.
Since a presentation is made in a style according to a boarding and alighting time, it is possible to present vehicles to an operator according to timing for monitoring. Accordingly, it is possible to present vehicles to be monitored in a more timely manner. For example, it is possible to present, as a vehicle to be preferentially monitored, a vehicle having a calculated time shorter than that of another vehicle.
Moreover, in the information processing method according to one aspect of the present disclosure, in each of the object detection processes, an object outside a moving vehicle and a position of the object are detected, and the style of the presentation is determined according to the object and the position of the object detected.
Conventionally, there is a technique for controlling the operation of a vehicle according to a state of an object outside the vehicle. However, the state of the object varies, and it is difficult to perform control assuming everything. On the other hand, it is conceivable that a monitoring operator checks the outside of a vehicle, constant monitoring of the outsides of vehicles increases the burden of the operator. In view of the above, with the present configuration, since a style of a presentation for monitoring is controlled according to objects outside vehicles, an operator can easily find out vehicles to be monitored. Accordingly, it is possible to reduce the burden of the operator while assuring safety for vehicles when objects are located outside vehicles.
Moreover, the information processing method according to one aspect of the present disclosure further includes: calculating, for each of the vehicles, at least one of a difficulty level of self-driving by the vehicle and a danger level outside the vehicle, using the object and the position of the object; and determining the style of the presentation according to the at least one of the difficulty level and the danger level calculated.
Since a presentation is made in a style according to at least one of difficulty levels of self-driving and danger levels outside vehicles, an operator can more accurately find out vehicles to be monitored. For example, it is possible to present, as a vehicle to be preferentially monitored, a vehicle having at least one of a relatively high difficulty level of self-driving and a relatively high danger level outside the vehicle.
Moreover, in the information processing method according to one aspect of the present disclosure, the presentation includes displaying an image showing at least one of an entrance and an exit of each of the vehicles.
For this reason, an operator can easily check safety at the time of boarding and alighting.
Moreover, in the information processing method according to one aspect of the present disclosure, the presentation includes displaying an image showing an outside of each of the vehicles.
For this reason, an operator can easily check safety outside a vehicle.
Moreover, in the information processing method according to one aspect of the present disclosure, the determining of the style of the presentation includes providing a higher visibility of the presentation for a higher priority level that is a priority level included in the priority levels.
Since a higher visibility is provided for a vehicle having a higher priority level, an operator can quickly check information regarding vehicles to be monitored.
Moreover, in the information processing method according to one aspect of the present disclosure, the determining of the style of the presentation includes providing a greater amount of information in the presentation for a higher priority level that is a priority level included in the priority levels.
Since a greater amount of information is provided for a vehicle having a higher priority level in a presentation, an operator can understand the details of information regarding vehicles to be monitored.
Further, since a less amount of information is provided for a vehicle having a lower priority level, it is possible to reduce frame rates of vehicles to be presented to the operator, and degrade image quality. For this reason, it is possible to reduce network load among devices, vehicles, and presentation devices that execute this information processing method.
Moreover, in the information processing method according to one aspect of the present disclosure, the determining of the style of the presentation includes emphasizing the presentation further for a higher priority level that is a priority level included in the priority levels.
Since a presentation is emphasized further for a vehicle having a higher priority level, an operator can quickly identify vehicles to be monitored.
Moreover, in the information processing method according to one aspect of the present disclosure, the determining of the style of the presentation includes starting the presentation when a priority level included in the priority levels exceeds a threshold value.
Since only a vehicle having a priority level higher than or equal to a threshold value is presented, an operator can focus on vehicles to be monitored.
Moreover, in the information processing method according to one aspect of the present disclosure, the determining of the style of the presentation includes arranging the presentation in order of the priority levels.
Since vehicles are presented in order of priority levels, an operator can quickly identify vehicles to be monitored. In addition, the operator can check a vehicle to be monitored next in advance, which can make monitoring more efficient.
Moreover, in the information processing method according to one aspect of the present disclosure, the priority levels are further determined based on at least one of positions of the vehicles, times corresponding to the positions of the vehicles, histories of required times for a boarding passenger to complete boarding and required times for an alighting passenger to complete alighting, and weather information corresponding to the positions of the vehicles.
With this, it is possible to set priority levels to vehicles more appropriately. For this reason, it is possible to cause a presentation device to present vehicles to be monitored more appropriately.
It should be noted that these general and specific aspects may be implemented using a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or any combination of systems, methods, integrated circuits, computer programs, or recording media.
Hereinafter, embodiments will be described in detail with reference to the drawings. It should be noted that each of the embodiments described hereinafter shows a specific example of the present disclosure. The numerical values, shapes, materials, structural components, the arrangement and connection of the structural components, steps, the processing order of the steps, etc., shown in the following embodiments are mere examples, and are therefore not intended to limit the present disclosure. Furthermore, among the structural components in the following embodiments, structural components not recited in any one of the independent claims defining the most generic concept of the present disclosure are described as optional structural components.
It should be noted that the figures are schematic diagrams and are not necessarily precise illustrations. Furthermore, in the figures, substantially identical components are assigned the same reference signs, and overlapping description thereof may be omitted or simplified.
In what follows, an information processing method, an information processing apparatus, and an information processing system according to each of the embodiments of the present disclosure will be described.

EMBODIMENT 1

(Configuration)

FIG. 1 is a block diagram ofinformation processing system 1 according toEmbodiment 1.
Information processing system 1 includesvehicle 3,information processing apparatus 5, andterminal 7.
Vehicle 3 is a self-driving vehicle that can be driven by remote control by a remote control operator thereof, and can self-drive when not remotely controlled. Examples ofvehicle 3 include a bus, a tax, and a train. The present embodiment assumes thatvehicle 3 is a bus and has stopped at a stop (bus stop). Hereinafter, a person who gets on or plans to get onvehicle 3 is referred to as a boarding passenger, and a person who gets off or plans to get offvehicle 3 is referred to as an alighting passenger. A term indicating both boarding passengers and alighting passengers is referred to as boarding and alighting passengers.
Vehicle 3 includes first obtaining unit (first obtainer) 31, second obtaining unit (second obtainer) 32, third obtaining unit (third obtainer) 33, fourth obtaining unit (fourth obtainer) 34, and communication unit (communicator) 35.
First obtainingunit 31 is a device that obtains information indicating boarding and alighting start times, and is, for example, a clock. Specifically, a boarding and alighting start time indicates a time at which a boarding and alighting passenger starts boarding or alighting fromvehicle 3 whenvehicle 3 has stopped at a stop. First obtainingunit 31 transmits the information indicating the boarding and alighting start times toinformation processing apparatus 5. It should be noted that a boarding and alighting start time may be a time at whichvehicle 3 is stopped at a stop by automated driving.
Second obtainingunit 32 obtains vehicle position information (e.g. global positioning system (GPS) information) indicating the current position ofvehicle 3. Second obtainingunit 32 transmits the obtained vehicle position information toinformation processing apparatus 5 andterminal 7. The current position ofvehicle 3 is an example of the position ofvehicle 3.
Third obtainingunit 33 is a device that obtains boarding and alighting passenger images that are captured images of boarding and alighting passengers boarding and alighting fromvehicle 3, and is, for example, a camera. Specifically, third obtainingunit 33 generates boarding and alighting passenger images by capturing people (e.g. boarding passengers) waiting at a stop ofvehicle 3 and people (e.g. alighting passengers) standing up invehicle 3. Third obtainingunit 33 transmits the boarding and alighting passenger images toinformation processing apparatus 5. Third obtainingunit 33 is an example of a sensor. Moreover, a boarding and alighting passenger image is an example of sensor data.
Fourth obtainingunit 34 is a device that obtains vehicle inside and outside images each showing the inside and outside ofvehicle 3, and is, for example, a camera. A vehicle inside and outside image includes vehicle's surroundings, that is, information about pedestrians, vehicles, bicycles, etc. outsidevehicle 3, information about passengers insidevehicle 3, and information about boarding and alighting passengers close to the entrance and exit ofvehicle 3. Fourth obtainingunit 34 transmits the vehicle inside and outside images toterminal 7. It should be noted that fourth obtainingunit 34 may be used for third obtainingunit 33, and boarding and alighting passenger images may be vehicle inside and outside images.
Communication unit 35 is a communication interface that transmits information indicating boarding and alighting start times, vehicle position information, and boarding and alighting passenger images toinformation processing apparatus 5, and transmits the vehicle position information and vehicle inside and outside images toterminal 7. In addition,communication unit 35 receives control commands for remote control whenvehicle 3 is remotely controlled by a remote control operator.Communication unit 35 outputs control commands to an actuator, an indicator, etc. (not shown) thatcontrol vehicle 3.
Here, there arevehicles 3, and each ofvehicles 3 has a vehicle ID for identifying itself.Communication unit 35 transmits information indicating boarding and alighting start times, vehicle position information, and boarding and alighting passenger images associated with the respective vehicle IDs toinformation processing apparatus 5, and transmits vehicle inside and outside images associated with the respective vehicle IDs toterminal 7.
Information processing apparatus 5 is a device capable of remotely controllingvehicles 3 that are self-driving, according to an instruction from an operator in a remote place.Information processing apparatus 5 includesdetection processor 51, determination unit (determiner) 52,storage controller 53,communication unit 54, andpresentation controller 55.
Detection processor 51 performs object detection processes. Examples of an object detection process include detecting at least one of boarding passengers and alighting passengers forvehicle 3 that has stopped, and detecting at least one of boarding passenger attributes or count and alighting passenger attributes or count. Specifically,detection processor 51 obtains boarding and alighting passenger images from third obtainingunit 33 of eachvehicle 3, detects a boarding and alighting passenger count, boarding and alighting passenger attributes, etc., based on the boarding and alighting passenger images, and generates boarding and alighting passenger information for eachvehicle 3. A boarding and alighting passenger attribute indicates information such as adult, child, and elderly.Detection processor 51 outputs boarding and alighting passenger information todetermination unit 52.
Determination unit 52 determines a style of a presentation for monitoring or controllingvehicles 3, using the results of the object detection processes. Styles of presentation will be described in detail later.
Moreover,determination unit 52 obtains boarding and alighting passenger information fromdetection processor 51, and obtains previous boarding and alighting data from a database stored instorage controller 53 whendetermination unit 52 obtains a boarding and alighting start time and vehicle position information from first obtainingunit 31 and second obtainingunit 32 of eachvehicle 3. Specifically,determination unit 52 requests, fromstorage controller 53, previous boarding and alighting data similar to the boarding and alighting passenger information, the boarding and alighting start time, and the vehicle position information, and obtains the similar previous boarding and alighting data fromstorage controller 53.
Determination unit 52 obtains previous boarding and alighting data fromstorage controller 53, and determines, forrespective vehicles 3, priority levels that are indicators formonitoring vehicles 3 on a priority basis. As shown inFIG. 2A,determination unit 52 transmits priority level information in which the determined priority levels are associated with the vehicle IDs ofvehicles 3 toterminal 7.FIG. 2A is a table illustrating priority level information according toEmbodiment 1.
Storage controller 53 stores previous boarding and alighting data of which a database is made. Boarding and alighting data is information such as boarding and alighting start times, stops, boarding and alighting passenger attributes, boarding and alighting passenger counts each for a different one of boarding and alighting passenger attributes, required times for all boarding and alighting passengers to complete boarding and alighting, and boarding and alighting times each corresponding to a different one of the boarding and alighting passenger attributes. In response to a request fromdetermination unit 52,storage controller 53 searches for previous boarding and alighting data similar to boarding and alighting passenger information, and transmits the similar previous boarding and alighting data todetermination unit 52. It should be noted thatstorage controller 53 may store a trained model trained using previous boarding and alighting data, instead of the previous boarding and alighting data of which the database is made. In this case,storage controller 53 transmits the trained model todetermination unit 52 in response to a request fromdetermination unit 52.
FIG. 2B and FIG. 2C show boarding and alighting data.FIG. 2B and FIG. 2C each are a table illustrating boarding and alighting data according toEmbodiment 1. As shown inFIG. 2B, the boarding and alighting data include "TIME" whenvehicle 3 stopped at a stop, "LOCATION" at whichvehicle 3 stopped, "BOARDING AND ALIGHTING PASSENGER ATTRIBUTE AND COUNT," and "REQUIRED TIME" for all boarding and alighting passengers to complete boarding and alighting. As shown inFIG. 2C, the boarding and alighting data further includes "BOARDING AND ALIGHTING TIME" per person in connection with each boarding and alighting passenger attribute.
As shown inFIG. 1,communication unit 54 is a communication interface that receives information indicating boarding and alighting start times, vehicle position information, and boarding and alighting passenger images, and transmits priority level information toterminal 7.Communication unit 54 is an example of an obtaining unit.
Presentation controller 55 controls a style of a presentation made byterminal 7 according to the style of the presentation determined bydetermination unit 52. In other words,presentation controller 55 outputs a command for controlling a presentation made byterminal 7 so thatterminal 7 makes the presentation in the style according to priority levels determined.
Terminal 7 is an operating terminal used by an operator to monitor the moving state ofvehicle 3 and remotely operatevehicle 3, and is, for example, a personal computer and a tablet terminal. Terminal 7 includes presentation unit (presenter) 71, operating unit (operator) 72,timer 73, andcommunication unit 74.
Presentation unit 71 is a monitor that presents vehicle inside and outside images, vehicle position information, etc. for eachvehicle 3. Specifically,presentation unit 71 presents, for eachvehicle 3,information regarding vehicle 3 such as a vehicle inside image, an entrance and exit image, an image of the outside of vehicle 3 (the surroundings of vehicle 3), a priority level, a boarding and alighting completion time, a vehicle ID and a stop.
Presentation unit 71 presentsinformation regarding vehicles 3 in various styles.Presentation unit 71 presents, for example, theinformation regarding vehicle 3 having the highest priority level with a large-area image, only theinformation regarding vehicle 3 having a priority level higher than or equal to a threshold value amongvehicles 3, theinformation regarding vehicle 3 having a high priority level in detail,information regarding vehicles 3 arranged in the order according to their priority levels with same-size images, andvehicle 3 having the highest priority level whose image is emphasized by being colored.
Operatingunit 72 is an input unit that receives remote control ofvehicle 3 by an operator. The operator checks whether all boarding and alighting passengers have boarded and alighted (check on completion of boarding and alighting), and operates operatingunit 72 whenvehicle 3 is caused to depart. Operatingunit 72 is, for example, a touch panel and a keyboard. Operatingunit 72 transmits control commands for remote control tovehicle 3 viacommunication unit 74.
Timer 73 is, for example, a clock that measures the time when the operator operates operatingunit 72 to transmit a control command for causingvehicle 3 to depart.
Communication unit 74 is a communication interface that transmits control commands tovehicle 3.
It should be noted that, in the present embodiment,timer 73 is not an essential component interminal 7, and second obtainingunit 32 is not an essential component invehicle 3. For this reason,timer 73 and second obtainingunit 32 need not be included, and any one oftimer 73 and second obtainingunit 32 may be provided.

(Operation)

Next, the following describes the operation of the information processing method,information processing apparatus 5, andinformation processing system 1 according to the present embodiment. Specifically, the following describes the operation ofinformation processing system 1 performed from whenvehicle 3 stops at a bus stop to whenvehicle 3 departs after a certain period of time.
FIG. 3 is a sequence diagram illustrating the operation ofinformation processing system 1 according toEmbodiment 1.
As shown inFIG. 3,vehicle 3 obtains information indicating a boarding and alighting start time, and transmits the obtained information to information processing apparatus 5 (step S101). In addition,vehicle 3 obtains vehicle position information indicating the current position ofvehicle 3, and transmits the obtained vehicle position information toinformation processing apparatus 5 and terminal 7 (step S102).Determination unit 52 ofinformation processing apparatus 5 receives the information indicating the boarding and alighting start time and the vehicle position information viacommunication unit 54.
Moreover,vehicle 3 obtains boarding and alighting passenger images that are captured images of boarding and alighting passengers boarding and alighting fromvehicle 3, and transmits the obtained boarding and alighting passenger images to information processing apparatus 5 (step S103).Detection processor 51 ofinformation processing apparatus 5 receives the boarding and alighting passenger images viacommunication unit 54.
In the mean time,vehicle 3 obtains vehicle inside and outside images, and transmits the obtained vehicle inside and outside images to terminal 7 (step S104).
Detection processor 51 generates boarding and alighting passenger information by detecting a boarding and alighting passenger count, boarding and alighting passenger attributes, etc., based on the boarding and alighting passenger images received in step S103 (step S105).Detection processor 51 outputs the boarding and alighting passenger information to determination unit 52 (step S106).
Determination unit 52 requests, fromstorage controller 53, previous boarding and alighting data similar to the obtained boarding and alighting passenger information, boarding and alighting start time, and vehicle position information (step S107).
In response to the request fromdetermination unit 52,storage controller 53 searches for the previous boarding and alighting data similar to the boarding and alighting passenger information, the boarding and alighting start time, and the vehicle position information, and outputs the similar previous boarding and alighting data to determination unit 52 (step S108). Boarding and alighting data includes information such as times at whichvehicle 3 stops at stops, stops, boarding and alighting passenger attributes, counts each for a different one of boarding and alighting passenger attributes, required times from when all boarding and alighting passengers start boarding and alighting to when the all boarding and alighting passengers finish boarding and alighting, boarding and alighting times each for a different one of the boarding and alighting passenger attributes, as shown inFIG. 2B and FIG. 2C. It should be noted thatstorage controller 53 may transmit a trained model todetermination unit 52 in response to a request fromdetermination unit 52, instead of outputting similar previous boarding and alighting data in response to a request fromdetermination unit 52.
Determination unit 52 obtains the previous boarding and alighting data fromstorage controller 53, and sets a priory level for monitoring for each vehicle 3 (step S109).Determination unit 52 determines a priority level for eachvehicle 3 according to the boarding and alighting passenger information, the boarding and alighting start time, and the vehicle position information. The priority levels will be described later.Determination unit 52 transmits, toterminal 7, priority level information in which the set priory levels are associated with the vehicle IDs (step S110). It should be noted thatdetermination unit 52 may obtain a trained model fromstorage controller 53 instead of obtaining previous boarding and alighting data fromstorage controller 53, and set a priority level for monitoring for eachvehicle 3. In this case,determination unit 52 determines the priority level for eachvehicle 3 using the obtained trained model.
Here,determination unit 52 determines thatinformation regarding vehicle 3 is to be presented byterminal 7 in a more highly visible style of a presentation as the priority level is higher, a style of a presentation in which an amount of information is greater as the priority level is higher, or a more emphasized style of a presentation as the priority level is higher. In addition,determination unit 52 determines thatinformation regarding vehicles 3 is to be presented in a style of a presentation in whichvehicles 3 are arranged in order of decreasing priority level.Presentation controller 55 controls the presentation made byterminal 7 according to a determined style, that is, causes terminal 7 to start presenting theinformation regarding vehicle 3 when the priority level exceeds a threshold value.
Next,terminal 7 receives the vehicle inside and outside image obtained in step S104 and the priority level information obtained in step S110. Terminal 7 makes a presentation according to the priority levels (step S111). Terminal 7 makes a presentation in various styles according to the priority levels.FIG. 4A to FIG. 4E each illustrate a style of a presentation made bypresentation unit 71.
FIG. 4A is a diagram illustrating a presentation made byterminal 7 according toEmbodiment 1. As shown inFIG. 4A,presentation unit 71 presentsvehicle 3 having the highest priority level with the large-area image (in a highly visible style). Specifically,presentation unit 71 presents onlyvehicles 3 having at least a predetermined priority level (threshold value) amongvehicles 3, andvehicle 3 having the highest priority level with a large-area image. Theinformation regarding vehicle 3 having the highest priority level indicates the required time for all passengers to complete boarding and alighting fromvehicle 3, theimages regarding vehicle 3, and the stop. Theimages regarding vehicle 3 include the vehicle inside image, the entrance and exit image, the vehicle outside images, etc. Moreover, for example,presentation unit 71 presents, forvehicles 3 having the second highest priority level or a lower priority level, information simpler than that ofvehicle 3 having the highest priority level. Specifically,presentation unit 71 presents only one kind ofimages regarding vehicles 3, such as images whose frame rates are reduced and images whose quality is reduced. Furthermore,presentation unit 71 does not presentinformation regarding vehicle 3 having a priority level lower than the predetermined priority level, that is,vehicle 3 having the sixth highest priority level or a lower priority level in the present embodiment.
FIG. 4B is a diagram illustrating a presentation made byterminal 7 according toEmbodiment 1. As shown inFIG. 4B,presentation unit 71 presents only theinformation regarding vehicles 3 having at least a predetermined priority level amongvehicles 3. InFIG. 4B,presentation unit 71 presents only theinformation regarding vehicles 3 having a similar boarding and alighting completion time on a priority basis. In the present embodiment,presentation unit 71 presents information regarding twovehicles 3 but not information regardingother vehicles 3 having the third highest priority level or a lower priority level.
FIG. 4C is a diagram illustrating a presentation made byterminal 7 according toEmbodiment 1. As shown inFIG. 4C,presentation unit 71 presents theinformation regarding vehicle 3 having the highest priority level with the large-area image, and onlyvehicles 3 having at least a predetermined priority level amongvehicles 3. InFIG. 4C,presentation unit 71 presents theimages regarding vehicles 3.
FIG. 4D is a diagram illustrating a presentation made byterminal 7 according toEmbodiment 1. As shown inFIG. 4D,presentation unit 71 presents all the images regarding vehicles 3 (in a style of a presentation in which an amount of information is increased) at a size that is large enough to allow an operator to view theinformation regarding vehicles 3. InFIG. 4D,presentation unit 71 presents theinformation regarding vehicles 3 arranged in order of decreasing priority level.
FIG. 4E is a diagram illustrating a presentation made byterminal 7 according toEmbodiment 1. As shown inFIG. 4E,presentation unit 71 presents theinformation regarding vehicle 3 having the highest priority level that is emphasized by being colored.
It should be noted that the presentations inFIG. 4A to FIG. 4E are mere examples, and the present embodiment is not limited to these. For example, a presentation includes displaying images each showing at least one of the entrance and the exit of a different one ofvehicles 3. In addition, a presentation includes images each showing the outside of a different one ofvehicles 3.
Here, we will return to the description of the sequence diagram shown inFIG. 3.
When all the passengers complete boarding and alighting fromvehicle 3,terminal 7 receives a departure instruction from the operator (step S112). For example, the operator checks on the completion of boarding and alighting viaterminal 7. Then, when the operator determines the completion of boarding and alighting, the operator inputs a departure instruction intoterminal 7. Terminal 7 receives the departure instruction inputted. It should be noted that, in the checking on the completion of boarding and alighting, terminal 7 may determine whether any passenger is close to the entrance and the exit by image recognition.
When terminal 7 receives the departure instruction from the operator, terminal 7 transmits to vehicle 3 a control command for causingvehicle 3 to depart (step S113).
Whenvehicle 3 receives the control command transmitted in step S113,vehicle 3 departs (step S114).
Moreover, when terminal 7 receives the departure instruction in step S112,timer 73 measures a boarding and alighting completion time in response to the reception of the departure instruction, and transmits information indicating the boarding and alighting completion time to information processing apparatus 5 (step S115).
Next,determination unit 52 stores intostorage controller 53 the information indicating the boarding and alighting start time received in step S101, and updates a database in storage controller 53 (step S118).
Moreover,determination unit 52 calculates a required time from the information indicating the boarding and alighting start time received in step S116 and the information indicating the boarding and alighting completion time stored in step S115, stores the required time intostorage controller 53, and updates the database in storage controller 53 (step S118). It should be noted thatdetermination unit 52 may update a trained model using boarding and alighting data such as times, locations, boarding and alighting passenger attributes, boarding and alighting passenger counts, and required times, instead of updating the above database instorage controller 53.
Furthermore,determination unit 52 stores intostorage controller 53 the boarding and alighting passenger information obtained in step S106 (step S117), and updates the database in storage controller 53 (step S118).
In this manner, the processing byinformation processing system 1 is ended. It should be noted that, ininformation processing system 1, the updating of the database and trained model instorage controller 53 in steps S115 to S118 is not an essential process, and such a process need not be performed.
Next, the following describes the operation ofinformation processing apparatus 5 with reference toFIG. 5. Here, processing is performed aftervehicle 3 stops at a predetermined position.FIG. 5 is a flow chart illustrating determination of a priority level by the operation ofinformation processing apparatus 5 according toEmbodiment 1.
First, as shown inFIG. 5,detection processor 51 ofinformation processing apparatus 5 receives boarding and alighting passenger images from each vehicle 3 (step S161).Detection processor 51 generates boarding and alighting passenger information by detecting a boarding and alighting passenger count, boarding and alighting passenger attributes, etc., based on the boarding and alighting passenger images.Detection processor 51 outputs the boarding and alighting passenger information todetermination unit 52.
Determination unit 52 ofinformation processing apparatus 5 receives fromvehicle 3 information indicating a boarding and alighting start time, and vehicle position information (step S162).
Determination unit 52 determines whether any boarding and alighting passenger is invehicle 3 based on the boarding and alight passenger information (step S163). Specifically,determination unit 52 determines whether the boarding and alighting passenger count included in the boarding and alighting passenger information is at least 1.
When the boarding and alighting passenger information indicates one or more boarding and alighting passengers (YES in step S163),determination unit 52 calculates a boarding and alighting completion time. Specifically,determination unit 52 calculates a required time from when all passengers start boarding and alighting to when they complete boarding and alighting, based on the boarding and alighting passenger information, previous boarding and alighting data, etc.
Determination unit 52 adds the calculated required time to the boarding and alighting start time obtained in step S162 to calculate the boarding and alighting completion time for each vehicle 3 (step S164).
Next,determination unit 52 sets higher priority levels tovehicles 3 in order of closeness of the boarding and alighting completion times to the current time.Determination unit 52 sets a priority level to each ofvehicles 3 everytime vehicle 3 stops at a stop, and updates the priority levels ofvehicles 3 in order of boarding and alighting completion time (step S165). The current time is an example of a time.
In contrast, when the boarding and alighting passenger information indicates no boarding and alighting passenger (NO in step S163),determination unit 52 sets the highest priority level tovehicle 3 determined to include no boarding and alighting passenger (step S167). It should be noted that when there arevehicles 3 including no boarding and alighting passenger,determination unit 52 sets allsuch vehicles 3 asvehicles 3 having the highest priority level.Determination unit 52 calculates a boarding and alighting completion time for each ofother vehicles 3 in the same manner as step S164, sets priority levels toother vehicles 3, and updates the priority levels of other vehicles 3 (step S168).
Next,determination unit 52 transmits information indicating the priority levels ofvehicles 3 toterminal 7 via communication unit 54 (step S166). In this manner, the processing byinformation processing apparatus 5 is ended. The same processing is performed at a predetermined position that is the next stop.
In what follows, step S164 of calculating a boarding and alighting completion time inFIG. 5 will be specifically described with reference toFIG. 6. FIG. 6 is a flow chart illustrating calculation of a boarding and alighting completion time using a database by the operation ofinformation processing apparatus 5 according toEmbodiment 1. It should be noted that a boarding and alighting completion time may be calculated using a trained model instead of a database.
First, as shown inFIG. 6,determination unit 52 refers to previous boarding and alighting data stored in storage controller 53 (step S171). Specifically,determination unit 52 requests fromstorage controller 53 previous boarding and alighting data similar to boarding and alighting passenger information, a boarding and alighting start time, and vehicle position information, and obtains the requested previous boarding and alighting data fromstorage controller 53.
Determination unit 52 determines whether the previous boarding and alighting data similar to the boarding and alighting passenger information, the boarding and alighting start time, and the vehicle position information is in storage controller 53 (step S172). For example, similar previous boarding and alighting data is previous boarding and alighting data similar to at least one of boarding and alighting passenger information, a boarding and alighting start time, and vehicle position information. Take a boarding and alighting start time for example, similar previous boarding and alighting data is temporally close to a boarding and alighting start time, such as within one hour etc. from the boarding and alighting start time.
When the previous boarding and alighting data similar to the boarding and alighting passenger information is determined to be in storage controller 53 (YES in step S172),determination unit 52 selects a required time (previous required time) for all passengers to complete boarding and alighting as the current required time for all passengers to complete boarding and alighting (step S173).
Determination unit 52 calculates a boarding and alighting completion time by adding the required time selected in step S173 to the boarding start time (step S174). Subsequently, the processing is finished.
In contrast, when the previous boarding and alighting data similar to the boarding and alighting passenger information is determined not to be in storage controller 53 (NO in step S172),determination unit 52 extracts, from previous boarding and alighting data stored instorage controller 53, boarding and alighting passenger attributes and boarding and alighting times each corresponding to a different one of the boarding and alighting passenger attributes (step S175).
Determination unit 52 calculates a required time for all passengers to complete boarding and alighting, based on step S175 (step S176).Determination unit 52 proceeds to step S174 and calculates a boarding and alighting completion time. Subsequently, the processing is finished.
As described above, in the information processing method,information processing apparatus 5, andinformation processing system 1, when an operator monitors stoppedvehicles 3, it is possible to calculate, for eachvehicle 3, a required time for all passengers to complete boarding and alighting, based on a boarding passenger count, an alighting passenger count, boarding passenger attributes, and alighting passenger attributes.Information processing apparatus 5 sets a higher priority level forvehicle 3 that requires a shorter boarding and alighting completion time, based on a boarding and alighting start time and a required time. Terminal 7 presents, for example,vehicle 3 having a high priority level with a large-area image, onlyvehicle 3 having a priority level higher than or equal to a threshold value,information regarding vehicles 3 arranged in the order according to their priority levels,information regarding vehicles 3 in detail, andvehicle 3 having the highest priority level whose image is emphasized by being colored. For this reason, the operator can monitoronly vehicles 3 presented byterminal 7, that is, easily find out the vehicles to be monitored. Accordingly, since the number ofvehicles 3 to be simultaneously monitored by the operator is reduced, it is possible to diminish the operator's risk for overlookingvehicles 3 to be monitored. As a result, it is possible to reduce the burden of the operator who monitorsvehicles 3 while assuring safety forvehicles 3 by the monitoring.

EMBODIMENT 2

(Configuration)

In the present embodiment, a case is described in whichvehicle 130 is assumed to be self-driving.
Unless otherwise specifically stated, the configurations of an information processing method,information processing apparatus 150, andinformation processing system 101 according to the present embodiment are the same asEmbodiment 1. The same reference signs are given to the same components, and the detailed description of the components is omitted.
FIG. 7 is a block diagram ofinformation processing system 101 according toEmbodiment 2.
As shown inFIG. 7,information processing system 101 includesvehicle 130,information processing apparatus 150, andterminal 170.
Vehicle 130 includes fifth obtaining unit (fifth obtainer) 135, sixth obtaining unit (sixth obtainer) 136, seventh obtaining unit (seventh obtainer) 137, andcommunication unit 138.
Fifth obtainingunit 135 is a device that obtains vehicle state information indicating the state ofvehicle 130, and is, for example, an electronic control unit (ECU). Vehicle state information includes information indicating the state ofvehicle 130, such as moving, temporary stop, emergency stop, and speed. Fifth obtainingunit 135 transmits vehicle state information toinformation processing apparatus 150 andterminal 170.
Sixth obtainingunit 136 obtains vehicle position information (e.g. GPS information) indicating the current position ofvehicle 130. Sixth obtainingunit 136 transmits the obtained vehicle position information toinformation processing apparatus 150 andterminal 170.
Seventh obtainingunit 137 performs object detection processes. The object detection processes are each performed to detect any object outside a moving vehicle, and the position of the object. Seventh obtainingunit 137 obtains sensor information indicating a passenger state insidevehicle 130, the position of an object outsidevehicle 130, etc. Seventh obtainingunit 137 is an image sensor such as a camera that captures images of the outside and inside ofvehicle 130, a distance sensor such as a laser imaging detection and ranging (LIDAR), etc. Seventh obtainingunit 137 transmits sensor information toinformation processing apparatus 150 andterminal 170. Examples of the object outsidevehicle 130 include a person walking ahead ofvehicle 130 and a bicycle behindvehicle 130. Seventh obtainingunit 137 is an example of a sensor.
Communication unit 138 is a communication interface that transmits vehicle state information, vehicle position information, and sensor information toinformation processing apparatus 150. In addition,communication unit 138 receives control commands for remote control whenvehicle 130 is remotely controlled by a remote control operator.
Detection processor 151 ofinformation processing apparatus 150 obtains sensor information from seventh obtainingunit 137 ofvehicle 130, and generates vehicle inside and outside information indicating a passenger state insidevehicle 130, an object outsidevehicle 130, etc., based on the sensor information.Detection processor 151 outputs the vehicle inside and outside information todetermination unit 152.
Whendetermination unit 152 ofinformation processing apparatus 150 obtains the vehicle inside and outside information fromdetection processor 151,determination unit 152 obtains previous vehicle data stored in a database instorage controller 153. Specifically,determination unit 152 requests fromstorage controller 153 previous vehicle data similar to the vehicle inside and outside information, and obtains the requested previous vehicle data fromstorage controller 153. It should be noted thatdetermination unit 152 may request from storage controller 153 a trained model created based on previous vehicle data stored in a database instorage controller 153, instead of the previous vehicle data, and may obtain the trained model fromstorage controller 153.
Determination unit 152 calculates at least one of a difficulty level of self-driving by eachvehicle 130 and a danger level outsidevehicle 130, based on sensor information, previous vehicle data, etc. A difficulty level of self-driving is a parameter indicating the ease of moving byvehicle 130. For example, whenvehicle 130 is moving on a narrow road orvehicle 130 is moving in rain, fog, etc., a difficulty level of self-driving is high. A danger level outsidevehicle 130 is a parameter indicating a possibility of an accident in an area in whichvehicle 130 is moving. For example, an area with danger of rushing out, an area under construction, etc. have a high danger level. Hereinafter, the present embodiment will be described using mainly a danger level. It should be noted thatdetermination unit 152 may calculate at least one of a difficulty level of self-driving by eachvehicle 130 and a danger level outsidevehicle 130 using an obtained trained model instead of sensor information, previous vehicle data, etc.
Storage controller 153 ofinformation processing apparatus 150 is a storage device that stores previous vehicle data. Vehicle data is information, such as a label, a location corresponding to the label, a recommended operation corresponding to the label, and a danger level corresponding to the label. A label is a descriptive text about a location, such as an area with danger of rushing out and under construction. In response to a request fromdetermination unit 152,storage controller 153 searches for previous vehicle data similar to vehicle inside and outside information, and transmits the similar previous vehicle data todetermination unit 152. It should be noted that a danger level is optionally changed, and varies depending on, for example, the number of people outsidevehicle 130. It should be noted thatstorage controller 153 may store a trained model trained based on previous vehicle data, instead of the previous vehicle data.
FIG. 8A, 8B, and FIG. 8C each are a table illustrating previous vehicle data according toEmbodiment 2. As shown inFIG. 8A, previous vehicle data includes "LOCATION" in which a dangerous situation has occurred, "LABEL" that is a descriptive text about a dangerous situation, "RECOMMENDED OPERATION" corresponding to a label, and "DANGER LEVEL" corresponding to a label. As shown inFIG. 8B, the previous vehicle data further includes "STATE" into whichvehicle 130 has gone, such as autonomous emergency braking (AEB) activation and temporary stop, "LABEL" that is a descriptive text corresponding to a state, "RECOMMENDED OPERATION" corresponding to a state, and "DANGER LEVEL" corresponding to a state. As shown inFIG. 8C, the previous vehicle data further includes "SITUATION" of the outside ofvehicle 130 whilevehicle 130 is moving, such as a pedestrian on a road and a bicycle behindvehicle 130, "LABEL" that is a descriptive text about a situation, "RECOMMENDED OPERATION" corresponding to a situation, and "DANGER LEVEL" corresponding to a situation.
As shown inFIG. 7,presentation unit 171 ofterminal 170 receives viacommunication unit 173information regarding vehicles 130, such as the priority levels, states, inside images, outside images, vehicle IDs, and current positions ofvehicles 130, and presents the received information for eachvehicle 130.
Presentation unit 171 ofterminal 170 presents, for example,vehicle 130 having the highest priority level with a large-area image,only vehicle 130 having a priority level higher than or equal to a threshold value amongvehicles 130,information regarding vehicles 130 in detail,information regarding vehicles 130 arranged in the order according to their priority levels with same-size images, andvehicle 130 having the highest priority level whose image is emphasized by being colored.

(Operation)

Next, the following describes the operation of the information processing method,information processing apparatus 150, andinformation processing system 101 according to the present embodiment. Specifically, the following describes the operation ofinformation processing system 101 performed whilevehicle 130 is moving.
FIG. 9 is a sequence diagram illustrating the operation ofinformation processing system 101 according toEmbodiment 2.
As shown inFIG. 9,vehicle 130 obtains and transmits vehicle state information toinformation processing apparatus 150 and terminal 170 (step S201).Determination unit 152 ofinformation processing apparatus 150 receives the vehicle state information viacommunication unit 154.
In addition,vehicle 130 obtains vehicle position information indicating the current position ofvehicle 130, and transmits the vehicle position information toinformation processing apparatus 150 and terminal 170 (step S202).Determination unit 152 ofinformation processing apparatus 150 receives the vehicle position information viacommunication unit 154.
Vehicle 130 obtains sensor information indicating a passenger state insidevehicle 130, vehicle's surroundings, that is, an object outsidevehicle 130, the position of the object, etc., and transmits the obtained sensor information toinformation processing apparatus 150 and terminal 170 (step S203).Determination unit 152 ofinformation processing apparatus 150 receives the sensor information viacommunication unit 154.
Detection processor 151 generates vehicle inside and outside information indicating the passenger state insidevehicle 130, the position of the object outsidevehicle 130, etc., based on the sensor information received in step S203 (step S204).Detection processor 151 outputs the vehicle inside and outside information to determination unit 152 (step S205).
Determination unit 152 requests fromstorage controller 153 previous vehicle data similar to the obtained vehicle inside and outside information (step S206).
In response to the request fromdetermination unit 152,storage controller 153 searches for the previous vehicle data similar to the vehicle inside and outside information, and transmits the similar previous vehicle data to determination unit 152 (step S207). Previous vehicle data is illustrated byFIG. 8A to FIG. 8C, and is information including, for example, a location, the location corresponding to a label, a recommended operation corresponding to the label, a danger level corresponding to the location, the state ofvehicle 130, and a situation corresponding to the location.
Determination unit 152 obtains the previous vehicle data fromstorage controller 153, and sets a priority level to eachvehicle 130, for monitoringvehicles 130 on a priority basis (step S208).Determination unit 152 sets the priority level based on a danger level. The determination of the priority level will be described later.Determination unit 152 transmits information indicating the set priority level ofvehicle 130 to terminal 170 (step S209).
Next, terminal 170 receives the vehicle state information obtained in step S201, the vehicle inside and outside information, and the information indicating the priority level ofvehicle 130 set in step S209.Terminal 170 makes a presentation according to the priority levels (step S210).
Terminal 170 makes a presentation in various styles according to the priority levels.FIG. 10A to FIG. 10F each illustrate a style of a presentation made bypresentation unit 171.
FIG. 10A is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10A,presentation unit 171 presentsvehicle 130 having the highest priority level with a large-area image. Specifically,presentation unit 171 presents onlyvehicles 130 having at least a predetermined priority level amongvehicles 130, andvehicle 130 having the highest priority level with a large-area image in detail. Theinformation regarding vehicle 130 having the highest priority level indicates the priority level, state, inside image, outside images, vehicle ID, current position, etc. ofvehicle 130. Moreover, for example,presentation unit 171 presents, forvehicle 130 having the second highest priority level or lower priority level, information simpler than that ofvehicle 130 having the highest priority level. Specifically,presentation unit 171 presents only one kind of animage regarding vehicle 130. Furthermore,presentation unit 171 does not presentinformation regarding vehicle 130 having a priority level lower than the predetermined priority level, that is,vehicle 130 having the sixth highest priority level or a lower priority level in the present embodiment.
FIG. 10B is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10B,presentation unit 171 presents only theinformation regarding vehicles 130 having at least a predetermined priority level amongvehicles 130. In the present embodiment,presentation unit 171 presents information regarding twovehicles 130 but not information regardingother vehicles 130 having the third highest priority level or a lower priority level.
FIG. 10C is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10C,presentation unit 171 presents theinformation regarding vehicles 130 having the highest priority level with the large-area images, and onlyvehicles 130 having at least a predetermined priority level amongvehicles 130. InFIG. 10C,presentation unit 171 presents every detail of theinformation regarding vehicles 130.
FIG. 10D is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10D,presentation unit 171 presents theinformation regarding vehicles 130 at a size that is large enough for an operator to view theinformation regarding vehicles 130. InFIG. 10D,presentation unit 171 presents theinformation regarding vehicles 130 arranged in order of decreasing priority level.
FIG. 10E is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10E,presentation unit 171 presents theinformation regarding vehicle 130 having the highest priority level that is emphasized by being colored.
FIG. 10F is a diagram illustrating a presentation made byterminal 170 according toEmbodiment 2. As shown inFIG. 10F, whenvehicles 130 are continuously moving in column,presentation unit 171 collectively presentsvehicles 130 assingle vehicle 130.
It should be noted that the presentations inFIG. 10A to FIG. 10F are mere examples, and the present embodiment is not limited to these.
Here, we will return to the description of the sequence diagram shown inFIG. 9.
Terminal 170 receives an operating instruction from the operator (step S211). For example, the operator checks on the operating status of eachvehicle 130 based on the presentation made byterminal 170. Then, the operator inputs the operating instruction toterminal 170. When terminal 170 receives the operating instruction from the operator, terminal 170 transmits a control command tovehicle 130 presented by terminal 170 (step S212). Here, the control command is a command for controlling the movement ofvehicle 130, such as an emergency stop, a temporary stop, slow down, and steering operation ofvehicle 130.
Whenvehicle 130 receives the control command transmitted in step S212,vehicle 130 performs an operation and display according to the control command (step S213). In this manner, the processing byinformation processing system 101 is ended.
Terminal 170 generates situation data indicating an evaluation according to the control command transmitted in step S212. For example, when driving at reduced speed is performed due to an act of rushing out at point A,terminal 170 generates situation data indicating a location as "point A," a label as "danger of rushing out," a recommended operation as "slow down," and a danger level as "high." It should be noted that situation data may be generated by the operator making an input operation viaoperating unit 172. In addition, terminal 170 may automatically generate situation data.Terminal 170 transmits the generated situation data to information processing apparatus 150 (step S214).Storage controller 153 ofinformation processing apparatus 150 receives the situation data viacommunication unit 154.Storage controller 153 stores the situation data and updates a database instorage controller 153.
Moreover,determination unit 152 generates, for example, movement information indicating a position at whichvehicle 130 has stopped, information about an object outsidevehicle 130 at the time, etc., based on the vehicle state information, vehicle position information, and sensor information, and outputs the movement information to storage controller 153 (step S215).
Storage controller 153 obtains the movement information and updates the database in storage controller 153 (step S216). In this manner, the processing byinformation processing system 101 is ended.
It should be noted that, ininformation processing system 101, the update of the database instorage controller 153 from step S214 to step S216 is not an essential process, and such a process need not be performed.
The following describes howinformation processing apparatus 150 sets a priority level in detail.FIG. 11 is a flow chart illustrating determination of a priority level by the operation ofinformation processing apparatus 150 according toEmbodiment 2. It should be noted that a priority level may be determined using a trained model instead of a database.
First, as shown inFIG. 11,information processing apparatus 150 obtains vehicle state information, sensor information, and vehicle position information (step S261).
Determination unit 152 ofinformation processing apparatus 150 determines whethervehicle 130 is abnormal based on the vehicle state information (step S262). Specifically,determination unit 152 determines whether the vehicle state information includes information indicating a highly dangerous operation whenvehicle 130 is moving, such as AEB activation.
Whenvehicle 130 is determined to be abnormal (YES in step S262),determination unit 152 sets the priority level ofvehicle 130 having transmitted the vehicle state information to be "high" (step S263).
Determination unit 152 transmits information indicating the priority level ofvehicle 130, which is set in step S263, to terminal 170 (step S264).
Whenvehicle 130 is determined not to be abnormal (NO in step S262),determination unit 152 refers to a database regarding areas stored in storage controller 153 (step S265). Specifically,determination unit 152 searches the database for a location (an area) in whichvehicle 130 is currently moving, based on the vehicle position information.
Determination unit 152 determines whether the area in whichvehicle 130 is currently moving includes a danger point, based on the vehicle position information (step S266). Examples of the danger point include a point with danger of rushing out and a point under construction.
When the area in whichvehicle 130 is currently moving is determined to include the danger point in the database (YES in step S266),determination unit 152 sets the priority level ofvehicle 130, to which the information has been transmitted in step S261, to be "high" or "medium" (step S267), and proceeds to step S264.
When the area in whichvehicle 130 is currently moving is determined not to include the danger point in the database (NO in step S266),determination unit 152 refers to a database regarding danger scenes stored instorage controller 153, based on the vehicle position information and the sensor information (step S268). Specifically,determination unit 152 searches the database in storage controller for the area in whichvehicle 130 is currently moving, based on the vehicle position information and the sensor information. Here, a danger scene is a scene in whichvehicle 130 is expected to face a danger in an area in whichvehicle 130 has difficulty self-driving. For example, a danger scene is a case in which a bicycle is moving on the same road asvehicle 130, a case in which a bicycle is approachingvehicle 130 from behind, and a case in which there are many blind spots due to on-street parking and there is the danger of rushing out.
Determination unit 152 determines whether the area in whichvehicle 130 is currently moving includes the danger scene, based on the vehicle position information and the sensor information (step S269).
When the area is determined to include the danger scene (YES in step S269),determination unit 152 sets the priority level ofvehicle 130, to which the information has been transmitted in step S261, to be "high" or "medium" (step S270), and proceeds to step S264.
When the area is determined not to include the danger scene (NO in step S269),determination unit 152 sets the priority level ofvehicle 130, to which the information has been transmitted in step S261, to be "low" (step S271), and proceeds to step S264. As described above,information processing apparatus 150 determines the priority level and ends the processing. Although there are three priority levels of "high," "medium," and "low" in the present embodiment, there may be two priority levels or at least four priority levels.
It should be noted that a danger level is thought to differ depending on an area in whichvehicle 130 is moving, the weather of the area, etc. Accordingly, for example, whenvehicle 130 is moving on a narrow road or rain, fog, etc. has started,determination unit 152 may set the priority level to be "high" in step S267 and step S270, the priority level to be "very high" in step S263, and the priority level to be "medium" in step S267.Determination unit 152 may obtain weather information about the area in whichvehicle 130 is moving from an external server viacommunication unit 154.
As above, in the information processing method,information processing apparatus 150, andinformation processing system 101, it is possible to calculate the danger level for eachvehicle 130 based on the sensor information indicating the internal state and external state ofvehicle 130 when the operator monitors movingvehicles 130.Information processing apparatus 150 presents, asvehicle 130 to be monitored on a priority basis,vehicle 130 having a higher outside vehicle danger level, based on the danger level.Terminal 170 presents:vehicle 130 having a high priority with the large-area image; onlyvehicle 130 having the priority level higher than or equal to the threshold value; detailed information regarding allvehicles 130;information regarding vehicles 130 as much as possible; andvehicle 130 having the highest priority level whose image is emphasized by being colored. For this reason, the operator can monitoronly vehicles 130 presented byterminal 170, that is, more accurately find outvehicles 130 to be monitored. As a result, it is possible to further assure safety for the vehicles by monitoring the vehicles.

OTHER VARIATIONS

Although the information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according toEmbodiments 1 and 2 of the present disclosure have been described above, the information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 are not limited toaforementioned Embodiments 1 and 2.
For example, in the information processing method,information processing apparatus 5, andinformation processing system 1 according toaforementioned Embodiment 1, the example is given in which the style of the presentation for monitoring or controllingvehicles 3 is determined according to the boarding and alighting passengers, a style of a presentation may be determined according to the congestion degrees ofvehicles 3. For example,detection processor 51 detects the numbers of passengers, the numbers of empty seats, etc. insidevehicles 3 from vehicle inside and outside images obtained by fourth obtainingunit 34, and generates congestion degree information.Determination unit 52 determines priority levels based on the congestion degree information.Presentation controller 55 controls a style of a presentation based on the priority levels.
Moreover, when the same priority level is determined forvehicles 3 and 130 in the information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according toaforementioned Embodiments 1 and 2,information processing apparatuses 5 and 150 may notifyvehicles 3 and 130 about the same priority level, and an announcement may be made invehicles 3 and 130 based on the notification.
Moreover, in the information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according toaforementioned Embodiments 1 and 2, a portion of an image presented in a style based on priority levels may be emphasized, the portion relating to information used to determine the priority levels, and the image may be presented in a distinguishable manner. Specifically, the information used to determine the priority levels may be a detected object. For example, any boarding and alighting passenger in an image may be highlighted inEmbodiment 1. Further, any pedestrian or vehicle in an image may be highlighted inEmbodiment 2. Examples of a style of highlighting may include adding a color, a pattern, or a frame to an object, and flashing of an object.
Moreover, each processing component included in the information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according toaforementioned Embodiments 1 and 2 is implemented as an LSI (large-scale integrated circuit) that is typically an integrated circuit. The integrated circuits may be implemented in a single chip individually, or in a single chip that includes some or all of them.
Moreover, the method of circuit integration is not limited to LSI. Integration may be realized with a specialized circuit or a general purpose processor. A field-programmable gate array (FPGA) for which programming can be performed after an LSI is fabricated or a reconfigurable processor capable of reconfiguring connections and settings of circuit cells of an LSI may be used, instead.
It should be noted that each of the structural components in each ofaforementioned Embodiments 1 and 2 may be configured in the form of an exclusive hardware produce, or may be implemented by executing a software program suitable for the structural component. Each of the structural components may be implemented by means of a program executing unit, such as a CPU or a processor, reading and executing the software program recorded on a recording medium such as a hard disk or a semiconductor memory.
Moreover, numbers in the above description are examples used for specifically describing the present disclosure, and Embodiments 1 and 2 of the present disclosure are not limited by such numbers.
Moreover, the block diagrams illustrate one example of the division of functional blocks: a plurality of functional blocks may be implemented as a single functional block, a single functional block may be broken up into a plurality of functional blocks, and part of one function may be transferred to another functional block. In addition, functions of a plurality of functional blocks having similar functions may be processed in parallel or by time-division by a single hardware or software product.
Furthermore, since the processing order of the steps in each flow chart is one example given for specifically describing the present disclosure, other processing orders may be adopted. In addition, a part of the steps may be performed simultaneously (in parallel) with another step.
The information processing methods,information processing apparatuses 5 and 150, andinformation processing systems 1 and 101 according to one or more aspects of the present disclosure have been described above based onEmbodiments 1 and 2, Embodiments 1 and 2 of the present disclosure are not limited to the one or more aspects. Although the present invention has been described and illustrated in detail, it is clearly understood that the same is by way of example only and is not to be taken by way of limitation, the scope of the present invention being limited only by the terms of the appended claims.
It should be noted that, as described inEmbodiments 1 and 2, machine learning may be used for the processes performed bydetection processors 51 and 151 anddetermination units 52 and 152. Examples of machine learning include: supervised learning that learns a relationship between input and output using teaching data in which a label (output information) is given to input information; unsupervised learning that builds a data structure from only unlabeled inputs; semi-supervised learning that makes use of both labeled and unlabeled data; and reinforcement learning that learns continuous actions capable of receiving the greatest reward, by receiving feedback (reward) for actions selected based on state observation results. Further, examples of a specific technique of machine learning include neural networks (including deep learning using multi-layer neural networks), genetic programming, decision trees, Bayesian networks, and support vector machines (SVMs). The present disclosure may use any one of the above specific examples.

INDUSTRIAL APPLICABILITY

The present disclosure is applicable to devices that remotely operate self-driving vehicles, terminals that present the states of self-driving vehicles, or systems including these devices. In particular, the present disclosure is applied to self-driving buses, self-driving taxis, and self-driving trains.

REFERENCE SIGNS LIST

1, 101: information processing system
3, 130: vehicle
5, 150: information processing apparatus
7, 170: terminal (presentation device)
33: third obtaining unit (sensor)
51, 151: detection processor
52, 152: determination unit
54, 154: communication unit (obtaining unit)
55, 155: presentation controller
137: seventh obtainer (sensor)

Claims

An information processing method, comprising:
obtaining (S201, S202, S203), from vehicles (130), sensor data obtained from sensors of the vehicles;
performing (S204) object detection processes each using a different one of the sensor data;
characterised by determining (S208) a style of a presentation for monitoring or controlling the vehicles by a remote operator that is an operator in a remote place, using results of the object detection processes; and
controlling (S208, S209) the presentation made to the remote operator by a presentation device according to the style determined.
The information processing method according to claim 1, further comprising:
determining (S208) priority levels of the vehicles based on the results of the object detection processes; and
determining (S210) the style of the presentation according to the priority levels.
The information processing method according to claim 2, further comprising:
obtaining states of the vehicles from the vehicles; and
determining the priority levels based on the results of the object detection processes and the states of the vehicles.
The information processing method according to any one of claims 1 to 3,
wherein in each of the object detection processes, at least one of a boarding passenger and an alighting passenger is detected for a stopped vehicle, and
the style of the presentation is determined according to the at least one of the boarding passenger and the alighting passenger detected.
The information processing method according to claim 4,
wherein in each of the object detection processes, an attribute or a count of at least one of the boarding passenger and the alighting passenger is detected, and the style of the presentation is determined according to the attribute or the count detected.
The information processing method according to claim 5, further comprising:
calculating a time for completion of at least one of the boarding passenger boarding the stopped vehicle and the alighting passenger alighting from the stopped vehicle, using the attribute or the count; and
determining the style of the presentation according to the time calculated.
The information processing method according to any one of claims 1 to 3,
wherein in each of the object detection processes, an object outside a moving vehicle and a position of the object are detected, and
the style of the presentation is determined according to the object and the position of the object detected.
The information processing method according to claim 7, further comprising:
calculating, for each of the vehicles, at least one of a difficulty level of self-driving by the vehicle and a danger level outside the vehicle, using the object and the position of the object; and
determining the style of the presentation according to the at least one of the difficulty level and the danger level calculated.
The information processing method according to any one of claims 4 to 6,
wherein the presentation includes displaying an image showing at least one of an entrance and an exit of each of the vehicles.
The information processing method according to claim 7 or 8,
wherein the presentation includes displaying an image showing an outside of each of the vehicles.
The information processing method according to claim 2 or 3,
wherein the determining of the style of the presentation includes providing a higher visibility of the presentation for a higher priority level that is a priority level included in the priority levels.
The information processing method according to any one of claims 2, 3, and 11,
wherein the determining of the style of the presentation includes providing a greater amount of information in the presentation for a higher priority level that is a priority level included in the priority levels.
The information processing method according to any one of claims 2, 3, 11, and 12,
wherein the determining of the style of the presentation includes starting the presentation when a priority level included in the priority levels exceeds a threshold value.
The information processing method according to any one of claims 2, 3, and 11 to 13,
wherein the determining of the style of the presentation includes arranging the presentation in order of the priority levels.
An information processing system (5), comprising:
an obtaining unit (31, 32, 33, 34) that obtains, from vehicles (3), sensor data obtained from sensors of the vehicles;
a detection processor (51) that performs object detection processes each using a different one of the sensor data;
characterised by a determination unit (52) that determines a style of a presentation for monitoring or controlling the vehicles by a remote operator that is an operator in a remote place, using results of the object detection processes; and
a presentation controller (55) that controls the presentation made to the remote operator by a presentation device (71) according to the style determined.