e-puck2 PC side development

e-puck2 main wiki

Robot configuration

This section explains how to configure the robot based on the communication channel you will use for your developments, thus you need to read only one of the following sections, but it would be better if you spend a bit of time reading them all in order to have a full understanding of the available configurations.

USB

The main microcontroller is initially programmed with a firmware that support USB communication.

If the main microcontroller isn't programmed with the factory firmware or if you want to be sure to have the last firmware on the robot, you need to program it with the last factory firmware by referring to section main microcontroller firmware update.

The radio module can be programmed with either the Bluetooth or the WiFi firmware, both are compatible with USB communication:

• Bluetooth: refer to section radio module firmware update
• WiFi: download the radio module wifi firmware (25.02.19) and then refer to section radio module firmware update

When you want to interact with the robot from the computer you need to place the selector in position 8 to work with USB.

Section PC interface gives step by step instructions on how to connect the robot with the computer via USB.

Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both USB and Bluetooth communication channels use the same protocol called advanced sercom v2, refer to section Communication protocol: BT and USB for detailed information about this protocol.

Bluetooth

The main microcontroller and radio module of the robot are initially programmed with firmwares that together support Bluetooth communication.

If the main microcontroller and radio module aren't programmed with the factory firmware or if you want to be sure to have the last firmwares on the robot, you need to program them with the last factory firmwares:

• for the main microcontroller, refer to section main microcontroller firmware update
• for the radio module, refer to section radio module firmware update

When you want to interact with the robot from the computer you need to place the selector in position 3 if you want to work with Bluetooth.

Section Connecting to the Bluetooth gives step by step instructions on how to accomplish your first Bluetooth connection with the robot.

Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both Bluetooth and USB communication channels use the same protocol called advanced sercom v2, refer to section Communication protocol: BT and USB for detailed information about this protocol.

WiFi

For working with the WiFi, the main microcontroller must be programmed with the factory firmware and the radio module must be programmed with a dedicated firmware (not the factory one):

• for the main microcontroller, refer to section main microcontroller firmware update
• radio module wifi firmware (25.02.19), for information on how to update the firmware refer to section radio module firmware update

Put the selector in position 15.

Section Connecting to the WiFi gives step by step instructions on how to accomplish your first WiFi connection with the robot.

The communication protocol is described in detail in the section Communication protocol: WiFi.

Connecting to the Bluetooth

The factory firmware of the radio module creates 3 Bluetooth channels using the RFcomm protocol when the robot is paired with the computer:

1. Channel 1, GDB: port to connect with GDB if the programmer is in mode 1 or 3 (refer to chapter Configuring the Programmer's settings for more information about these modes)
2. Channel 2, UART: port to connect to the UART port of the main processor
3. Channel 3, SPI: port to connect to the SPI port of the main processor (not yet implemented. Just do an echo for now)

By default, the e-puck2 is not visible when you search for it in the Bluetooth utility of your computer.
To make it visible, it is necessary to hold the USER button (also labeled "esp32" on the electronic board) while turning on the robot with the ON/OFF button.

Then it will be discoverable and you will be able to pair with it.
Note that a prompt could ask you to confirm that the number written on the screen is the same on the e-puck. just ignore this and accept. Otherwise if you are asked for a pin insert 0000.

Windows 7

When you pair your computer with the e-puck2, 3 COM ports will be automatically created. To see which COM port corresponds to which channel you need to open the properties of the paired e-puck2 robot from Bluetooth devices. Then the ports and related channels are listed in the Services tab, as shown in the following figure:

Windows 10

When you pair your computer with the e-puck2, 6 COM ports will be automatically created. The three ports you will use have Outgoing direction and are named e_puck2_xxxxx-GDB, e_puck2_xxxxx-UART, e_puck2_xxxxx-SPI. xxxxx is the ID number of your e-puck2.
To see which COM port corresponds to which channel you need to:

1. open the Bluetooth devices manager
2. pair with the robot
3. click on More Bluetooth options
4. the ports and related channels are listed in the COM Ports tab, as shown in the following figure:
   

Linux

Once paired with the Bluetooth manager, you need to create the port for communicating with the robot by issueing the command:
sudo rfcomm bind /dev/rfcomm0 MAC_ADDR 2
The MAC address is visible from the Bluetooth manager. The parameter 2 indicates the channel, in this case a port for the UART channel is created. If you want to connect to another service you need to change this parameter accordingly (e.g. 1 for GDB and 3 for SPI). Now you can use /dev/rfcomm0 to connect to the robot.

Mac

When you pair your computer with the e-puck2, 3 COM ports will be automatically created: /dev/cu.e-puck2_xxxxx-GDB, /dev/cu.e-puck2_xxxxx-UART and /dev/cu.e-puck2_xxxxx-SPI. xxxxx is the ID number of your e-puck2.

Testing the Bluetooth connection

You need to download the PC application provided in section PC interface: available executables.
In the connection textfield you need to enter the UART channel port, for example:

• Windows 7: COM258
• Windows 10: e_puck2_xxxxx-UART
• Linux: /dev/rfcomm0
• Mac: /dev/cu.e-puck2_xxxxx-UART

and then click Connect.
You should start receiving sensors data and you can send commands to the robot.

Alternatively you can also use a simple terminal program (e.g. realterm in Windows) instead of the PC application, then you can issue manually the commands to receive sensors data or for setting the actuators (once connected, type h + ENTER for a list of availables commands).

Connecting to multiple robots

Here is a simple python script multi-robot.py that open a connection with 2 robots and exchange data with them using the advanced sercom protocol. This example can be extended to connect to more than 2 robots.

Python examples

Here are some basic Python example that show how to get data from the robot through Bluetooth using the commands available with the advanced sercom v2:

• printhelp.py: print the list of commands available in the advanced sercom v2
• getprox.py: print the values of the proximity sensors

In both examples you need to set the correct Bluetooth serial port related to the robot.

Connecting to the WiFi

The WiFi channel is used to communicate with robot faster than with Bluetooth. At the moment a QQVGA (160x120) color image is transferred to the computer together with the sensors values at about 10 Hz; of course the robot is also able to receive commands from the computer.
In order to communicate with the robot through WiFi, first you need to configure the network parameters on the robot by connecting directly to it, since the robot is initially configured in access point mode, as explained in the following section. Once the configuration is saved on the robot, it will then connect automatically to the network and you can connect to it.

The LED2 is used to indicate the state of the WiFi connection:

• red indicates that the robot is in access point mode (waiting for configuration)
• green indicates that the robot is connected to a network and has received an IP address
• blue (toggling) indicates that the robot is transferring the image to the computer
• off when the robot cannot connect to the saved configuration

Network configuration

If there is no WiFi configuration saved in flash, then the robot will be in access point mode in order to let the user connect to it and setup a WiFi connection. The LED2 is red.

The access point SSID will be e-puck2_0XXXX where XXXX is the id of the robot; the password to connect to the access point is e-puck2robot.
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address 192.168.1.1. The available networks are scanned automatically and listed in the browser page as shown in figure 1. Choose the WiFi signal you want the robot to establish a conection with from the web generated list, and enter the related password; if the password is correct you'll get a message saying that the connection is established as shown in figure 2. After pressing OK you will be redirected to the main page showing the network to which you're connected and the others available nearby as shown in figure 3. If you press on the connected network, then you can see your IP address as shown in figure 4; take note of the address since it will be needed later.

[1]	[2]	[3]	[4]

Now the configuration is saved in flash, this means that when the robot is turned on it will read this configuration and try to establish a connection automatically.
Remember that you need to power cycle the robot at least once for the new configuration to be active.

Once the connection is established, the LED2 will be green.

In order to reset the current configuration you need to press the user button for 2 seconds (the LED2 red will turn on), then you need to power cycle the robot to enter access point mode.

Finding the IP address

Often the IP address assigned to the robot will remain the same when connecting to the same network, so if you took note of the IP address in section Network configuration you're ready to go to the next section.

Otherwise you need to connect the robot to the computer with the USB cable, open a terminal and connect to the port labeled Serial Monitor (see chapter Finding the USB serial ports used). Then power cycle the robot and the IP address will be shown in the terminal (together with others informations), as illustrated in the following figure:

Testing the WiFi connection

A dedicated WiFi version of the PC application was developed to communicate with the robot through TCP protocol. You can download the executable from one of the following links:

• Windows executable - WiFi
• Mac (not available yet)
• Ubuntu 14.04 (or later) - 64 bit

If you are interested to the source code, you can download it with the command git clone -b wifi --recursive https://github.com/e-puck2/monitor.git

Run the PC application, insert the IP address of the robot in the connection textfield and then click on the Connect button. You should start receiving sensors data and you can send commands to the robot. The LED2 blue will toggle.

Web server

When the robot is in access point mode you can have access to a web page showing the camera image and some buttons that you can use to move the robot; it is a basic example that you can use as a starting point to develop your own web browser interface.
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address 192.168.1.1/monitor.html.

Communication protocol

This section is the hardest part to understand. It outlines all the details about the communication protocols that you'll need to implement in order to communicate with the robot form the computer. So spend a bit of time reading and re-reading this section in order to grasp completely all the details.

Bluetooth and USB

The communication protocol is based on the advanced sercom protocol, used with the e-puck1.x robot. The advanced sercom v2 includes all the commands available in the advanced sercom protocol and add some additional commands to handle the new features of the e-puck2 robot. In particular here are the new commands:

Command	Description	Return value / set value
0x08	Get all sensors	see section Communication protocol: WiFi
0x09	Set all actuators	see section Communication protocol: WiFi
0x0A	Set RGB LEDs, values from 0 (off) to 100 (completely on)	[LED2_red][LED2_blue][LED2_green][LED4_red][LED4_blue][LED4_green][LED6_red][LED6_blue][LED6_green][LED8_red][LED8_blue][LED8_green]
0x0B	Get button state: 0 = not pressed, 1 = pressed	[STATE]
0x0C	Get all 4 microphones volumes	[MIC0_LSB][MIC0_MSB][MIC1_LSB][MIC1_MSB][MIC2_LSB][MIC2_MSB][MIC3_LSB][MIC3_MSB]
0x0D	Get distance from ToF sensor (millimeters)	[DIST_LSB][DIST_MSB]
0x0E	Get SD state: 0 = micro sd not connected, 1 = micro sd connected	[STATE]

WiFi

The communication is based on TCP; the robot create a TCP server and wait for a connection.

Each packet is identified by an ID (1 byte). The following IDs are used to send data from the robot to the computer:

• 0x00 = reserved
• 0x01 = QQVGA color image packet (only the first segment includes this id); packet size (without id) = 38400 bytes; image format = RGB565
• 0x02 = sensors packet; packet size (without id) = 104 bytes; the format of the returned values are based on the advanced sercom protocol and are compatible with e-puck1.x:

• Acc: raw axes values, between -1500 and 1500, resolution is +-2g
• Acceleration: acceleration magnitude , between 0.0 and about 2600.0 (~3.46 g)
• Orientation: between 0.0 and 360.0 degrees

  0.0 deg	90.0 deg	180 deg	270 deg

• Inclination: between 0.0 and 90.0 degrees (when tilted in any direction)

  0.0 deg	90.0 deg

• Gyro: raw axes values, between -32768 and 32767, range is +-250dps
• Magnetometer: raw axes values expressed in float, range is +-4912.0 uT (magnetic flux density expressed in micro Tesla)
• Temp: temperature given in Celsius degrees
• IR proximity: between 0 (no objects detected) and 4095 (object near the sensor)
• IR ambient: between 0 (strong light) and 4095 (dark)
• ToF distance: distance given in millimeters
• Mic volume: between 0 and 4095
• Motors steps: 1000 steps per wheel revolution
• Battery:
• uSD state: 1 if the micro sd is present and can be read/write, 0 otherwise
• TV remote data: RC5 protocol
• Selector position: between 0 and 15
• Ground proximity: between 0 (no surface at all or not reflective surface e.g. black) and 1023 (very reflective surface e.g. white)
• Ground ambient: between 0 (strong light) and 1023 (dark)
• Button state: 1 button pressed, 0 button released

• 0x03 = empty packet (only id is sent); this is used as an acknowledgment for the commands packet when no sensors and no image is requested

The following IDs are used to send data from the computer to the robot:

• 0x80 = commands packet; packet size (without id) = 20 bytes:

• request:
  • bit0: 0=stop image stream; 1=start image stream
  • bit1: 0=stop sensors stream; 1=start sensors stream
• settings:
  • bit0: 1=calibrate IR proximity sensors
  • bit1: 0=disable onboard obstacle avoidance; 1=enable onboard obstacle avoidance (not implemented yet)
  • bit2: 0=set motors speed; 1=set motors steps (position)
• left and right: when bit2 of settings field is 0, then this is the desired motors speed (-1000..1000); when 1 then this is the value that will be set as motors position (steps)
• LEDs: 0=off; 1=on; 2=toggle
  • bit0: 0=LED1 off; 1=LED1 on
  • bit1: 0=LED3 off; 1=LED3 on
  • bit2: 0=LED5 off; 1=LED5 on
  • bit3: 0=LED7 off; 1=LED7 on
  • bit4: 0=body LED off; 1=body LED on
  • bit5: 0=front LED off; 1=front LED on
• RGB LEDs: for each LED, it is specified in sequence the value of red, green and blue (0...100)
• sound id: 0x01=MARIO, 0x02=UNDERWOLRD, 0x04=STARWARS, 0x08=4KHz, 0x10=10KHz, 0x20=stop sound

For example to receive the camera image (stream) the following steps need to be followed:
1) connect to the robot through TCP
2) send the command packet:

0x80

0x01

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

0x00

3) read the ID (1 byte) and the QQVGA color image pakcet (38400 bytes)
4) go to step 3

Webots

TBD

ROS

This chapter explains how to use ROS with the e-puck2 robots by connecting them via Bluetooth to the computer that runs the ROS nodes. Basically all the sensors are exposed to ROS and you can also send commands back to the robot through ROS. Both Pyhton and cpp versions are implemented to give the user the possibility to choose its preferred programming language. Here is a general schema:
Click to enlarge

First of all you need to install and configure ROS, refer to http://wiki.ros.org/Distributions for more informations. This tutorial is based on ROS Kinetic.

Starting from the work done with the e-puck1 (see E-Puck ROS), we updated the code in order to support the e-puck2 robot.

Initial configuration

The following steps need to be done only once, after installing ROS:

1. If not already done, create a catkin workspace, refer to http://wiki.ros.org/catkin/Tutorials/create_a_workspace. Basically you need to issue the following commands:

  mkdir -p ~/catkin_ws/src
  cd ~/catkin_ws/src
  catkin_init_workspace
  cd ~/catkin_ws/
  catkin_make
  source devel/setup.bash 

2. You will need to add the line source ~/catkin_ws/devel/setup.bash to your .bashrc in order to automatically have access to the ROS commands when the system is started

3. Move to ~/catkin_ws/src and clone the ROS e-puck2 driver repo:

• if you are working with Python: git clone -b e-puck2 https://github.com/gctronic/epuck_driver
• if you are working with cpp: git clone -b e-puck2 https://github.com/gctronic/epuck_driver_cpp

4. Install the dependencies:

• Python:
  • The ROS e-puck2 driver is based on the e-puck2 Python library that requires some dependencies:
    • install the Python setup tools: sudo apt-get install python-setuptools
    • install the Python image library: sudo apt-get install python-imaging
    • install pybluez: sudo pip install pybluez
      • install pybluez dependencies: sudo apt-get install libbluetooth-dev
• cpp:
  • install the library used to communicate with Bluetooth: sudo apt-get install libbluetooth-dev

5. Open a terminal and go to the catkin workspace directory (~/catkin_ws) and issue the command catkin_make, there shouldn't be errors

6. Program the e-puck2 robot with the factory firmware and put the selector in position 3

Running the Python ROS node

First of all get the last version of the ROS e-puck2 driver from github. Move to ~/catkin_ws/src and issue: git clone -b e-puck2 https://github.com/gctronic/epuck_driver.
Then build the driver by opening a terminal and issueing the command catkin_make from within the catkin workspace directory (e.g. ~/catkin_ws).
Moreover make sure the node is marked as executable by opening a terminal and issueing the following command from within the catkin workspace directory (e.g. ~/catkin_ws): chmod +x ./src/epuck_driver/scripts/epuck2_driver.py.

Before actually starting the e-puck2 node you need to configure the e-puck2 robot as Bluetooth device in the system, refer to section Connecting to the Bluetooth.
Once the robot is paired with the computer, you need to take note of its MAC address (this will be needed when launching the ROS node). To know the MAC address of a paired robot, go to System Settings, Bluetooth and select the robot; once selected you'll see in the right side the related MAC address.

First thing to do before launching the script file is running the roscore, open another terminal tab and issue the command roscore.

Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on roslaunch).
Open a terminal and issue the following command: roslaunch epuck_driver epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F'.
B4:E6:2D:EB:9C:4F is the e-puck2 Bluetooth MAC address that need to be changed accordingly to your robot.

If all is going well you'll see the robot make a blink meaning it is connected and ready to exchange data and rviz will be opened showing the informations gathered from the topics published by the e-puck2 driver node.
The following graph shows all the topics published by the e-puck2 driver node and the rviz interface:
Click to enlarge Click to enlarge

Move the robot

You have some options to move the robot.

The first one is to use the rviz interface: in the bottom left side of the interface there is a Teleop panel containing an interactive square meant to be used with differential drive robots. By clicking in this square you'll move the robot, for instance by clicking on the top-right section, then the robot will move forward-right.

The second method to move the robot is using the

Control the RGB LEDs

The general command to change the RGB LEDs colors is the following:
rostopic pub -1 /rgb_values sensor_msgs/ChannelFloat32 "{name: rgb, values: [LED2 red, LED2 green, LED2 blue, LED4 red, LED4 green, LED4 blue, LED6 red, LED6 green, LED6 blue, LED8 red, LED8 green, LED8 blue]}"
The values range is from 0 (off) to 100 (completely on). Have a look at the e-puck2 overview to know the position of the RGB LEDs.
For instance to set all the RGB LEDs to red, issue the following command:
rostopic pub -1 /rgb_values sensor_msgs/ChannelFloat32 "{name: rgb, values: [100.0,0.0,0.0, 100.0,0.0,0.0, 100.0,0.0,0.0, 100.0,0.0,0.0]}"
To turn off all the RGB LEDs issue the following command:
rostopic pub -1 /rgb_values sensor_msgs/ChannelFloat32 "{name: rgb, values: [0.0,0.0,0.0, 0.0,0.0,0.0, 0.0,0.0,0.0, 0.0,0.0,0.0]}"