Repository files navigation

This project documents set up of Reinforcement learning environment for flight control. This will enable each user/learner/student/flight enthusiast to come with their own RL algorithm that can interact with and control in real time an aircraft in a simulation environment. The simulator used isX-Plane (the flight environment). This gives a realistic environment to work with together with parameters to help keep the simulation as close to reality as possible.

Update: Use the leaderboard wiki Page-->(Leaderboard) to add details of your own scenario or (reinforcement learning) algorithm.

  GYM-XPLANE ENVIRONMENT INSTALLATION

To install the gym x-palne environment go through the read me file of the folder :Gym-Xplane-final versionThis is the latest version of the actual gym-xplane environment.

The other folders contain additional modules for additional functionalities. The names of such folders would givean idea as to what each of such modules does.

Read the Wiki to get a general idea of the environment and training procedure (Gym-Xplane Wiki).

Requirements

• X Plane (Demo version) (main requirement)
  • XPLane
• XPlaneConnect (replace this with different python udp connection file if required)
  • XPlaneConnect [https://github.com/nasa/XPlaneConnect]
• RL Algorithm (define your own RL algorithm or use standard baslines. )
  • Tensorforce
  • RLLIB
  • OpenAI Baselines
• Lua Programming/FlyWithLua (for seamless interaction with XPlane)

General setup

• GYM XPLANE ENVIRONMENT
• MULTI AGENT ENVIRONMENT SET UP (Work in progress)

The possible action parameters are given below. The choice of parameters would depend on the task. For example one might decide that the last three parameters are not important for the keep heading task (Gym Xplane Wiki) .

The state space parameters are well documented inXPlane Data Ref. The number and configuration of state space parameters would also depend on the task. It is possible to use a derived state spade parameters.

Note that, usually a UDP connection (XPlaneConnect) is required to read any parameter parameter from XPlane. An example of how to read parameter from UDP connection is shown below:

client = xpc.XPlaneConnect() # UDP connectorclient.getDREF("sim/flightmodel/position/P")[0][0] # moment Pclient.getPOSI() # get the lat, long,altitude, pitch, roll, heading, gear

A typical state space parameter configuration is shown below:

The range of each parameter value would also depend on the configuration or simulation task.

client (UDP connector) already has a functiongetPOSI that helps read thelatitude, longitude, altitude, pitch, roll, heading. Other parameters could be added by using the clientgetDREF function. Note that the string"sim/flightmodel/position/P" is gotten from the Xplane Dataref referenced earlier (XPlane Data Ref)

To learn more about this go to :FlightDataVizDashBoard

• Use XVFB for fake display.
• Docker support (under test)
• Algorithm implementations (high priority) / GPU support to be included
• Real Time flight and data visualization interface.
• Investigate Federated Learning / Thompson Sampling / policy-space response oracles (psro)