Repository files navigation

This application helps visualize and interact with Spot robot's GraphNav maps. It consists of:

1. Flask Backend: Processes and serves GraphNav map data
2. React Frontend: Provides an interactive 2D visualization of the map along with recorded waypoint snapshots(images) and Retrieval Augmented Graph (RAG) database

Raw Map Visualization

Object Filtered Map

Waypoint Details with Camera Images

1. Install requirements:

   pipinstallflaskflask-corsopencv-pythonnumpyscipybosdyn-clientpillow

2. Run the Flask server:

   cdbackendpythonapp.py--map-path/path/to/map--rag-path/path/to/rag_db--port5000

Inmap-path, we expect a folder containing GraphNav map files:

• graph
• edge_snapshots
• waypoint_snapshotsInrag-path, we expect a folder containing the RAG database file:
• index.faiss
• index.pkl
• metadata_waypoint_id.json for each waypointTo build such a database, please refer to the main project repository atSpottyAI

1. Navigate to the React app directory:

   cd frontend

2. Install dependencies:

   npm install

3. Start the development server:

   npm start

If there is an error, you might have to set the following:

export NODE_OPTIONS=--openssl-legacy-provider

The application should be accessible athttp://localhost:3000

• 2D Map Visualization: Interactive canvas-based rendering with pan and zoom
• Waypoint Selection: Click on waypoints to view details and camera images
• Object Filtering: Filter waypoints by visible objects
• Waypoint Labeling: Edit waypoint labels with automatic updates
• Multiple Views: Toggle between anchored and non-anchored coordinate frames

• Python 3.8.20
• Node v22.13.0
• npm 10.9.2
• React 19.0.0
• MacOS 12.4

It would be great to add the following features:

• Path planning and visualization:
  • A* path planning
  • Path visualization on the map
• Real-time updates with WebSockets and Spot SDK