Movatterモバイル変換

1NASAVERVE: Interactive 3DVisualization within EclipseNASAVERVE: Interactive 3DVisualization within EclipseTamar Cohen and Mark B.AllanIntelligent Robotics GroupNASA Ames Research Center

2NASAVERVE: Interactive 3DVisualization within EclipseThe Intelligent Robotics Group (IRG)at NASA Ames Research Centerdevelops robotic rovers as researchplatforms for autonomous navigation,human robot interaction, and robotsoftware. Often we send a small team ofscientists and robot engineers into theﬁeld with some rovers, and science andengineering backroom teams remotelyoperate the rovers.It is necessary to visualize where therover is, what algorithms it is executing,and what scientiﬁc data it is gathering. K10 Red at Haughton Crater, Devon Island CA

3NASAVERVE: Interactive 3DVisualization within EclipseWe have created a number of tools to assist with remotesituational awareness. Some of these are web tools, and quite afew are Eclipse RCP applications, based aroundVERVE. VERVE is an integrated 3D view built on top of Ardor3D, an opensource 3D Java library.Concept of Operations

4NASAVERVE: Interactive 3DVisualization within EclipseFlight Control Room at Ames during HMP experiment

5NASAVERVE: Interactive 3DVisualization within EclipseScience backroom at Ames

6NASAVERVE: Interactive 3DVisualization within EclipseWe develop and work with many different robots for different needs, andattach various scientiﬁc instruments to them. We also collaborate withdifferent NASA centers to support remote operation of their robots.IRG’s K-series rovers feature:•  four-wheel drive•  all wheel steering •  GPS system•  digital compass•  stereo hazard cameras•  inertial measurement unit•  LIDARK-series rovers run RoverSoftware, which supportsautonomous navigation andobstacle avoidance.Krex at Basalt Hill, CA

7NASAVERVE: Interactive 3DVisualization within EclipseA simple model of K10Red; wekeep the models small and efficientand only articulate what will give ususeful information.TheVERVE 3DView is an Eclipse viewwith contents rendered by theArdor3D libraries which we havewrapped in a plug-in. We render theArdor3D canvas in an EclipseView.Ardor3D provides user interfacesupport within the view, ie overlaytext or floating images / areas.Ardor3D provides hooks for mousecontrol and keyboard input, alongwith typical controls for a 3D graphicslibrary (ie cameras, scene, lighting,etc). It supports standard 3D fileformats.

8NASAVERVE: Interactive 3DVisualization within EclipseThe Ardor3D scene graph is comprised of Spatials to deﬁne geometry and rendering settingsNodes spatials with parents and childrenTo build up the scene graph, you simply call node.attachChild(Spatial child).For each type and instance of a robot, we create a RobotNode (extends Node)which contains nodes that represent its model (3D object) and child nodes forrepresentation of scientiﬁc data.We have a reference between our conceptual representation of a robot and each ofits parts, and the Ardor3D nodes which represent each concept.public class RobotNode extends Node {!!final AbstractRobot m_robot;!!RobotModelNode m_model;!Node m_concepts;!Node m_sensors;!Node m_base;!!public RobotNode(AbstractRobot robot, Node model) {!super(robot.getName());!m_robot = robot;!setRobotModel(model);!}!

9NASAVERVE: Interactive 3DVisualization within Eclipse/**!* interface for scenegraph -> telemetry glue for robot visualization parts!*/!public interface IRobotPart {!public String getPartName();!public AbstractRobot getRobot();!!public void connectTelemetry() throws TelemetryException;!public void disconnectTelemetry() throws TelemetryException;!!public void attachToNodesIn(Node model) throws TelemetryException,IllegalStateException;!!public void handleFrameUpdate(long currentTime);!public void reset();!!public boolean isVisible();!public void setVisible(boolean state);!}!We have created a simple interface to connect theincoming telemetry to the robot parts that arerendered:

10NASAVERVE: Interactive 3DVisualization within EclipseDesert RATS 2011 featured several robots, vehicles,and astronauts with sensors

11NASAVERVE: Interactive 3DVisualization within EclipseA model of a space suit with instrument backpack and theCentaur 2 rover, which we worked with at Desert RATS

12NASAVERVE: Interactive 3DVisualization within EclipseBlack Point Lava Flow as a GeoTIFF with levels of detail, rendered with haze

13NASAVERVE: Interactive 3DVisualization within EclipseWe have a standard Eclipse viewwhich includes a tree populated withthe contents of the scene graph. Thistree can be extremely deep.Since elements are dynamically addedto and removed from the 3D scene,the tree must be populated withWeakReferences to support garbagecollection.When various events happen, the treerefreshes asynchronously.Checkboxes show and hide 3Dmodels in the Ardor3D view bysetting their render state.Scene Graph Viewif (show){!!spatial.setCullHint(CullHint.Inherit);} else {!!spatial.setCullHint(CullHint.Always);!}!

14NASAVERVE: Interactive 3DVisualization within EclipseWe use Java reﬂection and SWTdatabinding to automaticallygenerate SWT UI componentsbased on the Java objects deﬁnedin the robot class; manipulatingthese widgets affects how thecomponents are rendered in theArdor3D scene.This allows our robot engineersand scientists to customize thevisualization based on what theyare looking for.

15NASAVERVE: Interactive 3DVisualization within EclipseTypically we work with rovers toexplore terrain that has not beenwell mapped. We may have satelliteimagery and information about theterrain. This data is at a lowresolution and may not beaccurate. As the rovers traversethe area, they build more accuratemaps. We use LIDAR, a remote sensingtechnology that measures distancewith a laser. We use LIDAR bothfor a line scan and to return apoint cloud. We also do stereoreconstruction with our twohazard cameras.LIDAR

16NASAVERVE: Interactive 3DVisualization within EclipseRepresentation of LIDAR terrain inVERVEWhen we get LIDAR data backto our Eclipse RCP applicationfrom the rover, we can “groundtruth” or adjust the terrain tothe discovered data.

17NASAVERVE: Interactive 3DVisualization within EclipseYou can see the line scan and point cloud data that Centaur 2 hascaptured as it drives across a parking lot. Red arrows show the pathit has taken.

18NASAVERVE: Interactive 3DVisualization within EclipseWe use DDS, the Data Distribution Service for Real-Time Systems as ourmiddleware. By having this standard, we can create IDL (Interface DefinitionLanguage) files that describe the data which is being serialized betweenpublishers and subscribers. We define data formats for commands and fortelemetry, and various consumers (robots, software applications) can subscribeto the published topics. We configure DDS to handle poor connectivity and dropouts, since we aresimulating space missions which will have these issues.Adhering to the DDS standard makes it more straightforward to integrate withvarious robots and scientific instruments.Surface TeleroboticsWorkbench RCPApplicationK10Rover DDSGroundISSWe used DDS to have astronauts on the International Space Station control our K10Rover at NASA Ames

19NASAVERVE: Interactive 3DVisualization within EclipseTo facilitate collaboration between NASA centers, we came upwith a standard called RAPID (Robot API Delegate), built on top ofDDS. This includes a set of IDL ﬁles and Java libraries for commoncommands and telemetry.http://rapid.nasa.gov/

20NASAVERVE: Interactive 3DVisualization within EclipseFor our RAPID VERVE implementation, we have a baseclass to connect robot parts to RAPID telemetry.public abstract class RapidRobotPart extends AbstractRobotPart implementsIRapidMessageListener!{!…!@Override!public void connectTelemetry() throws TelemetryException {!if(isTelemetryEnabled()) {!try {for(MessageType msgType : rapidMessageTypes()){!!RapidMessageCollector.instance().addRapidMessageListener(getParticipantId(),!! ! ! ! ! !getRapidRobot().getAgent(), msgType, this );!This way, when the telemetry comes in, the relevantdata can be cached and when it is time to rerender,handleFrameUpdate is called.

21NASAVERVE: Interactive 3DVisualization within Eclipse!Our very simple render update thread runs as follows:Once the system and display are ready (until the display is disposed)Check the time that the last change occurred; if the elapsed time is enough, asynchronously run an update!timer = new Timer(); // an Ardor3D timer which uses System.nanoTime!!frameHandler = new FrameHandler(timer);!!frameHandler.addUpdater(new LogicalLayerUpdater());!!!// we expose camera controls in the Eclipse UI!!frameHandler.addUpdater(new CameraControlUpdater()); !!!renderUpdateThread = new RenderUpdateThread(applicationPlugin,! ! ! ! ! ! ! ! ! ! ! ! ! ! ! frameHandler, timer);!!renderUpdateThread.start();!frameHandler.updateFrame();We have data coming in too fast to render, so we throttle that back:

22NASAVERVE: Interactive 3DVisualization within Eclipse!public void handleFrameUpdate(long currentTime) {!if(isDirty()) {!try {!float[] ja = m_angleData;!JointInfo joint;!for(int i = 0; i < m_joints.length; i++) {!if(i < m_angleData.length) {!joint = m_joints[i];!if( joint.spatial != null && !Float.isNaN(ja[i]) ) {!m_rot.fromAngleNormalAxis(ja[i]+joint.offset,! ! ! ! ! ! !joint.rotAxis);!joint.spatial.setRotation(m_rot);!}!}!}!}!catch(Throwable t) {!logger.log(t);!}!setDirty(false);!}!}!Sample handleFrameUpdate for joint data

23NASAVERVE: Interactive 3DVisualization within EclipseScientiﬁc VisualizationHere we are rendering data coming back from a Ground PenetratingRadar (GPR) system, so scientists can analyze information about thesubstrate. They can customize the rendering of the GPR telemetry.Arrows indicateprevious position,and the ringsindicate the suntracker and themagnetic compass.

24NASAVERVE: Interactive 3DVisualization within EclipseWe also work with the SPHERES onthe International Space Station (ISS);we run DDS on an AndroidSmartphone which we connect to theSPHERES, and view and control it fromour Eclipse RCP Application. In our experiment in 2013, wesimulated an internal inspection of anISS module .We have commanded it from theground and astronauts havecommanded it from the ISS.

25NASAVERVE: Interactive 3DVisualization within EclipseScreenshot of the SPHERES Smartphone Workbench: Manual Control

26NASAVERVE: Interactive 3DVisualization within EclipseThe Surface Telerobotics experiment in 2013 examined howastronauts in the ISS can remotely operate a surface robot (K10Rover) across short time delays. We simulated an astronautteleoperating a rover on the lunar farside to deploy a low radiofrequency telescope. Deployment of a Telescope

27NASAVERVE: Interactive 3DVisualization within EclipseRoverscape at NASA Ameswith K10 Red

28NASAVERVE: Interactive 3DVisualization within EclipseRover running a Route Plan; Panorama coming in

29NASAVERVE: Interactive 3DVisualization within EclipseManually moving the rover forward and inspecting the ﬁlm

30NASAVERVE: Interactive 3DVisualization within EclipseDuring our Surface Telerobotics experiment, we worked with a 3rdparty company (TRACLabs) to subscribe to DDS messages providinginformational alerts about the state of our rover. We displayed thesein a growl-like overlay in theVERVE 3DView. These alerts are built ontop of Ardor3D's UI components and allows hooks to Eclipse UI.

31NASAVERVE: Interactive 3DVisualization within EclipseSurface Telerobotics Experiment in action:Luca Parmitano on the ISS operating K10 at Ames

32NASAVERVE: Interactive 3DVisualization within EclipseVERVE open source release:http://sourceforge.net/projects/irg-verve/Ardor3D in Java:http://www.ardor3d.comGoo Technologies, Ardor3D in #

33NASAVERVE: Interactive 3DVisualization within EclipseUpcoming IRG Work•  We have been doing a huge push to migrate our infrastructureto use maven, Nexus and git.•  We are just beginning migration to Eclipse E4•  We continue to develop free-ﬂying robots for the ISS•  We are working with KRex2, preparing to look for water in theMojave in the fall as part of the Volatile Prospector efforts•  http://www.astrobotic.com/2013/12/02/resource-aware-planning-for-robot-operations-at-the-lunar-pole/•  This Mojave work is in support of the RESOLVE project, todevelop a new set of instruments to use in the search for water•  http://www.spacenews.com/article/civil-space/39307nasa-planning-for-mission-to-mine-water-on-the-moon

34NASAVERVE: Interactive 3DVisualization within EclipseIntelligent Robotics Group at NASA Ames Research Center •  K10 Rover among others•  SPHERES•  xGDS Ground Data Systems•  VERVE 3D within Eclipse•  Contributed the moon to Google Earth•  Mars-o-vision (mars.planetary.org)•  GigaPan robotic camera•  GeoCam disaster response•  Ames Stereo Pipeline•  Vision Workbench•  Tensegrity research… and more!http://irg.arc.nasa.gov

Movatterモバイル変換

Change Language

NASA VERVE: Interactive 3D Visualization within Eclipse

Embed presentation

Recommended

More Related Content

Viewers also liked

Similar to NASA VERVE: Interactive 3D Visualization within Eclipse

Recently uploaded

NASA VERVE: Interactive 3D Visualization within Eclipse