Difference between pages "e-puck2" and "e-puck2 PC side development"

From GCtronic wiki
(Difference between pages)
Jump to: navigation, search
(Firmware update)
 
(Multiple robots)
 
Line 1: Line 1:
=Hardware=
+
[{{fullurl:e-puck2}} e-puck2 main wiki]<br/>
==Overview==
 
<span class="plainlinks">[http://www.gctronic.com/doc/images/e-puck2-overview.png <img width=500 src="http://www.gctronic.com/doc/images/e-puck2-overview_small.png">]</span>
 
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-features.png <img width=600 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-features_small.png">]</span><br/>
 
  
The following figures show the main components offered by the e-puck2 robot and where they are physically placed:<br/>
+
=Robot configuration=
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/epuck2-components-position.png <img width=800 src="http://projects.gctronic.com/epuck2/wiki_images/epuck2-components-position_small.png">]</span><br/>
+
This section explains how to configure the robot based on the communication channel you will use for your developments, thus you need to read only one of the following sections, but it would be better if you spend a bit of time reading them all in order to have a full understanding of the available configurations.
  
==Specifications==
+
==USB==
The e-puck2 robot maintains full compatibility with its predecessor e-puck (e-puck HWRev 1.3 is considered in the following table):
+
The main microcontroller is initially programmed with a firmware that support USB communication.<br/>
{| border="1"
+
 
|'''Feature'''
+
If the main microcontroller isn't programmed with the factory firmware or if you want to be sure to have the last firmware on the robot, you need to program it with the last factory firmware by referring to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update main microcontroller firmware update].<br/>
|'''e-puck1.3'''
+
 
|'''e-puck2'''
+
The radio module can be programmed with either the <code>Bluetooth</code> or the <code>WiFi</code> firmware, both are compatible with USB communication:
|'''Compatibility'''
+
* Bluetooth: refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update_2 radio module firmware update]
|'''Additional'''
+
* WiFi: download the [http://projects.gctronic.com/epuck2/esp32-firmware-wifi_25.02.19_e2f4883.zip radio module wifi firmware (25.02.19)] and then refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update_2 radio module firmware update]
|-  
+
 
|Size, weight
+
When you want to interact with the robot from the computer you need to place the selector in position 8 to work with USB. <br/>
|70 mm diameter, 55 mm height, 150 g
+
 
|Same form factor: 70 mm diameter, 45 mm, 130 g
+
Section [http://www.gctronic.com/doc/index.php?title=e-puck2#PC_interface PC interface] gives step by step instructions on how to connect the robot with the computer via USB.<br/>
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
 
|No e-jumper required
+
Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both USB and Bluetooth communication channels use the same protocol called [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB advanced sercom v2], refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB_2 Communication protocol: BT and USB] for detailed information about this protocol.<br/>
|-
+
 
|Battery, autonomy
+
==Bluetooth==
|LiIPo rechargeable battery (external charger), 1800 mAh. <br/>About 3 hours autonomy. Recharging time about 2-3h.
+
The main microcontroller and radio module of the robot are initially programmed with firmwares that together support Bluetooth communication.<br/>
|Same battery; USB charging, recharging time about 2.5h.
+
 
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
If the main microcontroller and radio module aren't programmed with the factory firmware or if you want to be sure to have the last firmwares on the robot, you need to program them with the last factory firmwares:
|USB charging
+
* for the main microcontroller, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update main microcontroller firmware update]
|-
+
* for the radio module, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update_2 radio module firmware update]
|Processor
+
 
|16-bit dsPIC30F6014A @ 60MHz (15 MIPS), DSP core for signal processing
+
When you want to interact with the robot from the computer you need to place the selector in position 3 if you want to work with Bluetooth. <br/>
|32-bit STM32F407 @ 168 MHz (210 DMIPS), DSP and FPU, DMA
+
 
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
Section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_Bluetooth Connecting to the Bluetooth] gives step by step instructions on how to accomplish your first Bluetooth connection with the robot.<br/>
|~10 times faster
+
 
|-
+
Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both Bluetooth and USB communication channels use the same protocol called [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB advanced sercom v2], refer to section [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB Communication protocol: BT and USB] for detailed information about this protocol.<br/>
|Memory
+
 
|RAM: 8 KB; Flash: 144 KB
+
==WiFi==
|RAM: 192 KB; Flash: 1024 KB
+
For working with the WiFi, the main microcontroller must be programmed with the factory firmware and the radio module must be programmed with a dedicated firmware (not the factory one):
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
* for the main microcontroller, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update main microcontroller firmware update]
|RAM: 24x more capable<br/>Flash:~7x more capable
+
* [http://projects.gctronic.com/epuck2/esp32-firmware-wifi_25.02.19_e2f4883.zip radio module wifi firmware (25.02.19)], for information on how to update the firmware refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update_2 radio module firmware update]
|-
+
Put the selector in position 15.<br/>
|Motors
+
 
|2 stepper motors with a 50:1 reduction gear; 20 steps per revolution; about 0.13 mm resolution
+
Section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_WiFi Connecting to the WiFi] gives step by step instructions on how to accomplish your first WiFi connection with the robot.<br/>
|Same motors
+
 
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
The communication protocol is described in detail in the section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#WiFi_2 Communication protocol: WiFi].<br/>
|
+
 
|-
+
=Connecting to the Bluetooth=
|Wheels
+
 
|Wheels diamater = 41 mm <br/>Distance between wheels = 53 mm
+
The factory firmware of the radio module creates 3 Bluetooth channels using the RFcomm protocol when the robot is paired with the computer:
|Same wheels
+
# Channel 1, GDB: port to connect with GDB if the programmer is in mode 1 or 3 (refer to chapter [http://www.gctronic.com/doc/index.php?title=e-puck2_programmer_development#Configuring_the_Programmer.27s_settings Configuring the Programmer's settings] for more information about these modes)
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
# Channel 2, UART: port to connect to the UART port of the main processor
|
+
# Channel 3, SPI: port to connect to the SPI port of the main processor (not yet implemented. Just do an echo for now)
|-
+
 
|Speed
+
By default, the e-puck2 is not visible when you search for it in the Bluetooth utility of your computer.<br>
|Max: 1000 steps/s (about 12.9 cm/s)
+
'''To make it visible, it is necessary to hold the USER button (also labeled "esp32" on the electronic board) while turning on the robot with the ON/OFF button.'''<br>
|Max: 1200 steps/s (about 15.4 cm/s)
+
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-bt-pair.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-bt-pair-small.png">]</span><br/>
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
Then it will be discoverable and you will be able to pair with it.<br>
|20% faster
+
Note that a prompt could ask you to confirm that the number written on the screen is the same on the e-puck. just ignore this and accept. Otherwise if you are asked for a pin insert 0000.
|-
+
 
|Mechanical structure
+
==Windows 7==
|Transparent plastic body supporting PCBs, battery and motors
+
When you pair your computer with the e-puck2, 3 COM ports will be automatically created.
|Same mechanics
+
To see which COM port corresponds to which channel you need to open the properties of the paired e-puck2 robot from <code>Bluetooth devices</code>. Then the ports and related channels are listed in the <code>Services</code> tab, as shown in the following figure:<br/>
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/BT-connection-win7.png <img width=300 src="http://projects.gctronic.com/epuck2/wiki_images/BT-connection-win7.png">]</span>
|
+
 
|-
+
==Windows 10==
|Distance sensor
+
When you pair your computer with the e-puck2, 6 COM ports will be automatically created. The three ports you will use have <code>Outgoing</code> direction and are named <code>e_puck2_xxxxx-GDB</code>, <code>e_puck2_xxxxx-UART</code>, <code>e_puck2_xxxxx-SPI</code>. <code>xxxxx</code> is the ID number of your e-puck2.<br/>
|8 infra-red sensors measuring ambient light and proximity of objects up to 6 cm
+
To see which COM port corresponds to which channel you need to:
|Same infra-red sensors <br/>Front real distance sensor, Time of fight (ToF), up to 2 meter.
+
# open the Bluetooth devices manager
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
# pair with the robot
|ToF sensor
+
# click on <code>More Bluetooth options</code>
|-
+
# the ports and related channels are listed in the <code>COM Ports</code> tab, as shown in the following figure:<br/>
|IMU
+
:<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/BT-connection-win10.png <img height=300 src="http://projects.gctronic.com/epuck2/wiki_images/BT-connection-win10.png">]</span>
|3D accelerometer and 3D gyro
+
 
|3D accelerometer, 3D gyro, 3D magnetometer
+
==Linux==
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
Once paired with the Bluetooth manager, you need to create the port for communicating with the robot by issueing the command: <br/>
|3D magnetometer
+
<code>sudo rfcomm bind /dev/rfcomm0 MAC_ADDR 2</code><br/>
|-
+
The MAC address is visible from the Bluetooth manager. The parameter <code>2</code> indicates the channel, in this case a port for the <code>UART</code> channel is created. If you want to connect to another service you need to change this parameter accordingly (e.g. <code>1</code> for <code>GDB</code> and <code>3</code> for <code>SPI</code>). Now you can use <code>/dev/rfcomm0</code> to connect to the robot.
|Camera
+
 
|VGA color camera; typical use: 52x39 or 480x1
+
==Mac==
|Same camera; typical use: 160x120
+
When you pair your computer with the e-puck2, 3 COM ports will be automatically created: <code>/dev/cu.e-puck2_xxxxx-GDB</code>, <code>/dev/cu.e-puck2_xxxxx-UART</code> and <code>/dev/cu.e-puck2_xxxxx-SPI</code>. xxxxx is the ID number of your e-puck2.
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
 
|Bigger images handling
+
==Testing the Bluetooth connection==
|-
+
You need to download the PC application provided in section [http://www.gctronic.com/doc/index.php?title=e-puck2#Available_executables PC interface: available executables].<br/>
|Audio
+
In the connection textfield you need to enter the UART channel port, for example:
|3 omni-directional microphones for sound localization<br/>speaker capable of playing WAV or tone sounds
+
* Windows 7: <code>COM258</code>
|4 omni-directional microhpones (digital) for sound localization<br/>speaker capable of playing WAV or tone sounds
+
* Windows 10: <code>e_puck2_xxxxx-UART</code>
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
* Linux: <code>/dev/rfcomm0</code>
| +1 front microphone
+
* Mac: <code>/dev/cu.e-puck2_xxxxx-UART</code>
|-
+
and then click <code>Connect</code>. <br/>
|LEDs
+
You should start receiving sensors data and you can send commands to the robot.<br/>
|8 red LEDs around the robot, green body light, 1 strong red LED in front
+
 
|4 red LEDs and 4 RGB LEDs around the robot, green light, 1 strong red LED in front
+
Alternatively you can also use a simple terminal program (e.g. <code>realterm</code> in Windows) instead of the PC application, then you can issue manually the commands to receive sensors data or for setting the actuators (once connected, type <code>h + ENTER</code> for a list of availables commands).
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
 
|4x RGB LEDs
+
==Python examples==
|-
+
Here are some basic Python examples that show how to get data from the robot through Bluetooth using the commands available with the [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB advanced sercom v2]:
|Communication
+
* [http://projects.gctronic.com/epuck2/printhelp.py printhelp.py]: print the list of commands available in the [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Bluetooth_and_USB advanced sercom v2]
|RS232 and Bluetooth 2.0 for connection and programming
+
* [http://projects.gctronic.com/epuck2/getprox.py getprox.py]: print the values of the proximity sensors
|USB Full-speed, Bluetooth 2.0, BLE, WiFi
+
* [http://projects.gctronic.com/epuck2/complete.py complete.py]: set all the actuators and get all the sensors data printing their values on the screen
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
* [http://projects.gctronic.com/epuck2/getimage.py getimage.py]: request an image and save it to disk
|WiFi, BLE
+
In all the examples you need to set the correct Bluetooth serial port related to the robot.
|-
+
 
|Storage
+
===Connecting to multiple robots===
|Not available
+
Here is a simple Python script [http://projects.gctronic.com/epuck2/multi-robot.py multi-robot.py] that open a connection with 2 robots and exchange data with them using the [http://www.gctronic.com/doc/index.php/Advanced_sercom_protocol advanced sercom protocol]. This example can be extended to connect to more than 2 robots.
|Micro SD slot
+
 
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
==C++ remote library==
|Micro SD
+
A remote control library implemented in C++ is available to control the e-puck2 robot via a Bluetooth connection from the computer.<br/>
|-
+
The remote control library is multiplatform and uses only standard C++ libraries.<br/>
|Remote Control
+
You can download the library with the command <code>git clone https://github.com/e-puck2/e-puck2_cpp_remote_library</code>.<br/>
|Infra-red receiver for standard remote control commands
+
A simple example showing how to use the library is also available; you can download it with the command <code>git clone https://github.com/e-puck2/e-puck2_cpp_remote_example</code>.<br/>
|Same receiver
+
Before building the example you need to build the library. Then when building the example, make sure that both the library and the example are in the same directory, that is you must end up with the following directory tree:<br>
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
: e-puck2_projects
|
+
::|_ e-puck2_cpp_remote_library
|-
+
::|_ e-puck2_cpp_remote_example
|Switch / selector
+
The complete API reference is available in the following link [http://projects.gctronic.com/epuck2/e-puck2_cpp_remote_library_api_reference_rev3ac41e3.pdf e-puck2_cpp_remote_library_api_reference.pdf].
|16 position rotating switch
+
 
|Same selector
+
=Connecting to the WiFi=
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
The WiFi channel is used to communicate with robot faster than with Bluetooth. At the moment a QQVGA (160x120) color image is transferred to the computer together with the sensors values at about 10 Hz; of course the robot is also able to receive commands from the computer.<br/>
|
+
In order to communicate with the robot through WiFi, first you need to configure the network parameters on the robot by connecting directly to it, since the robot is initially configured in access point mode, as explained in the following section. Once the configuration is saved on the robot, it will then connect automatically to the network and you can connect to it.
|-
+
 
|Extensions
+
The LED2 is used to indicate the state of the WiFi connection:
|Ground sensors, range and bearing, RGB panel, Gumstix extension, omnivision, your own
+
* red indicates that the robot is in ''access point mode'' (waiting for configuration)
|All extension supported
+
* green indicates that the robot is connected to a network and has received an IP address
|style="text-align:center;" | <img width=40 src="http://www.gctronic.com/doc/images/ok.png">
+
* blue (toggling) indicates that the robot is transferring the image to the computer
|
+
* off when the robot cannot connect to the saved configuration
|-
+
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-wifi-led.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-wifi-led-small.png">]</span><br/>
|Programming
+
 
|Free C compiler and IDE, Webots simulator, external debugger
+
==Network configuration==
|Free C compiler and IDE, Webots simulator, onboard debugger (GDB)
+
If there is no WiFi configuration saved in flash, then the robot will be in ''access point mode'' in order to let the user connect to it and setup a WiFi connection. The LED2 is red.
|style="text-align:center;" | <img width=30 src="http://www.gctronic.com/doc/images/plus.png">
+
 
|Onboard debugger
+
The access point SSID will be <code>e-puck2_0XXXX</code> where <code>XXXX</code> is the id of the robot; the password to connect to the access point is <code>e-puck2robot</code>.<br/>
|}
+
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address <code>192.168.1.1</code>. The available networks are scanned automatically and listed in the browser page as shown in ''figure 1''. Choose the WiFi signal you want the robot to establish a conection with from the web generated list, and enter the related password; if the password is correct you'll get a message saying that the connection is established as shown in ''figure 2''. After pressing <code>OK</code> you will be redirected to the main page showing the network to which you're connected and the others available nearby as shown in ''figure 3''. If you press on the connected network, then you can see your IP address as shown in ''figure 4''; <b>take note of the address since it will be needed later</b>.<br/>
 +
 
 +
<span class="plainlinks">
 +
<table>
 +
<tr>
 +
<td align="center">[1]</td>
 +
<td align="center">[2]</td>
 +
<td align="center">[3]</td>
 +
<td align="center">[4]</td>
 +
</tr>
 +
<tr>
 +
<td>[http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup1.png <img width=150 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup1.png">]</td>
 +
<td>[http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup2.png <img width=150 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup2.png">]</td>
 +
<td>[http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup3.png <img width=150 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup3.png">]</td>
 +
<td>[http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup4.png <img width=150 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup4.png">]</td>
 +
</tr>
 +
</table>
 +
</span><br/>
 +
Now the configuration is saved in flash, this means that when the robot is turned on it will read this configuration and try to establish a connection automatically.<br/>
 +
Remember that you need to power cycle the robot at least once for the new configuration to be active.<br/>
  
This is the overall communication schema:<br/>
+
Once the connection is established, the LED2 will be green.<br/>
<span class="plainlinks">[http://www.gctronic.com/doc/images/comm-overall-e-puck2E.jpg <img width=700 src="http://www.gctronic.com/doc/images/comm-overall-e-puck2E.jpg">]</span><br/>
 
  
==Documentation==
+
In order to reset the current configuration you need to press the user button for 2 seconds (the LED2 red will turn on), then you need to power cycle the robot to enter ''access point mode''.
* '''Main microcontroller''': STM32F407, [http://projects.gctronic.com/epuck2/doc/STM32F407xx_datasheet.pdf datasheet], [http://projects.gctronic.com/epuck2/doc/STM32F407_reference-manual.pdf reference-manual]
+
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-wifi-reset.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-wifi-reset-small.png">]</span><br/>
* '''Programmer/debugger''': STM32F413, [http://projects.gctronic.com/epuck2/doc/STM32F413x_datasheet.pdf datasheet], [http://projects.gctronic.com/epuck2/doc/STM32F413_reference-manual.pdf reference-manual]
 
* '''Radio module''': Espressif ESP32, [http://projects.gctronic.com/epuck2/doc/esp32_datasheet_en.pdf datasheet], [http://projects.gctronic.com/epuck2/doc/esp32_technical_reference_manual_en.pdf reference-manual]
 
* '''Camera''': PixelPlus PO8030D CMOS image sensor, [http://projects.gctronic.com/E-Puck/docs/Camera/PO8030D.pdf datasheet], no IR cut filter
 
: From about July 2019, the camera mounted on the e-puck2 robot is the Omnivision OV7670 CMOS image sensor, [http://projects.gctronic.com/epuck2/doc/OV7670.pdf datasheet]
 
* '''Microphones''': STM-MP45DT02, [http://projects.gctronic.com/epuck2/doc/mp45dt02.pdf datasheet]
 
* '''Optical sensors''': Vishay Semiconductors Reflective Optical Sensor, [http://projects.gctronic.com/epuck2/doc/tcrt1000.pdf datasheet]
 
* '''ToF distance sensor''': STM-VL53L0X, [http://projects.gctronic.com/epuck2/doc/VL53L0X-Datasheet.pdf datasheet], [http://projects.gctronic.com/epuck2/doc/VL53L0X-UserManual-API.pdf user-manual]
 
* '''IMU''': InvenSense MPU-9250, [http://projects.gctronic.com/epuck2/doc/MPU-9250-product-specification.pdf product-specification], [http://projects.gctronic.com/epuck2/doc/MPU-9250-Register-Map.pdf register-map]
 
* '''Motors''': [http://www.e-puck.org/index.php?option=com_content&view=article&id=7&Itemid=9 details]
 
* '''Speaker''': Diameter 13mm, power 500mW, 8 Ohm, DS-1389 or PSR12N08AK or similar
 
* '''IR receiver''': TSOP36230
 
  
==Migrating from e-puck1.x to e-puck2==
+
==Finding the IP address==
The e-puck2 robot maintains full compatibility with its predecessor e-puck, but there are some improvements that you should be aware of.<br/>
+
Often the IP address assigned to the robot will remain the same when connecting to the same network, so if you took note of the IP address in section [http://www.gctronic.com/doc/index.php?title=e-puck2#Network_configuration Network configuration] you're ready to go to the next section. <br/>
  
First of all the e-jumper, that is the small board that is attached on top of the e-puck1.x, isn't anymore needed in the e-puck2. The components available on the e-jumper are integrated directly in the robot board. On top of the e-puck2 you'll see a quite big free connector, this is used to attach the extensions board designed for the e-puck1.x that are fully compatible with the e-puck2; you must not connect the e-jumper in this connector.<br/>
+
Otherwise you need to connect the robot to the computer with the USB cable, open a terminal and connect to the port labeled <code>Serial Monitor</code> (see chapter [http://www.gctronic.com/doc/index.php?title=e-puck2#Finding_the_USB_serial_ports_used Finding the USB serial ports used]). Then power cycle the robot and the IP address will be shown in the terminal (together with others informations), as illustrated in the following figure:<br/>
+
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup5.png <img width=500 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-wifi-setup5.png">]</span>
Secondly you don't need anymore to unplug and plugin the battery for charging, but instead you can charge the battery (up to 1 ampere) directly by connecting the USB cable. If you want you can still charge the battery with the e-puck1.x external charger, in case you have more than one battery.<br/>
 
  
Moreover you don't need anymore a special serial cable (with probably an RS232 to USB adapter) to be able to communicate with the robot, but you can use the USB cable. Once connected to the computer a serial port will be available that you can use to easily exchange data with the robot.
+
==Testing the WiFi connection==
 +
A dedicated WiFi version of the PC application was developed to communicate with the robot through TCP protocol. You can download the executable from one of the following links:
 +
* [http://projects.gctronic.com/epuck2/monitor_wifi_27dddd4.zip Windows executable - WiFi]
 +
* Mac (not available yet)
 +
* [http://projects.gctronic.com/epuck2/monitor_wifi_linux64bit_27dddd4.tar.gz Ubuntu 14.04 (or later) - 64 bit]
  
==Extensions==
+
If you are interested to the source code, you can download it with the command <code>git clone -b wifi --recursive https://github.com/e-puck2/monitor.git</code><br/>
All the extensions (ground sensors, range and bearing, RGB panel, gumstix and omnvision) are supported by the e-puck2 robot, this means that if you have some extensions for the e-puck1.x you can still use them also with e-puck2.<br/>
 
For more information about using the gumstix extension with e-puck2 robot refer to [http://www.gctronic.com/doc/index.php?title=Overo_Extension#e-puck2 http://www.gctronic.com/doc/index.php?title=Overo_Extension#e-puck2].
 
  
=Getting Started=
+
Run the PC application, insert the IP address of the robot in the connection textfield and then click on the <code>Connect</code> button. You should start receiving sensors data and you can send commands to the robot. The LED2 blue will toggle.<br/>
The e-puck2 robot features 3 chips onboard:
 
* the main microcontroller, that is responsible for handling the sensors and actuators and which runs also the demos/algorithms
 
* the programmer, that provides programming/debugging capabilties and moreover it configures the USB hub and is responsible for the power management (on/off of the robot and battery measure)
 
* radio module, that is responsible for handling the wireless communication (WiFi, BLE, BT), the RGB LEDs and the user button (the RGB LEDs and button are connected to the radio module due to the pin number limitation on the main microcontroller)
 
  
The robot is shipped with the last firmware version programmed on all 3 chips, so you can immediately start using the robot.<br/>
+
==Web server==
The following sections explain the basic usage of the robot, <b>all the users should read this chapter completely in order to have a minimal working system ready to play with the e-puck2 robot</b>. Some sections will have more detailed information that can be read by following the links provided.
+
When the robot is in ''access point mode'' you can have access to a web page showing the camera image and some buttons that you can use to move the robot; it is a basic example that you can use as a starting point to develop your own web browser interface.<br/>
 +
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address <code>192.168.1.1/monitor.html</code>.
  
When required, dedicated informations are given for all platforms (Windows, Linux, Mac). The commands given for Linux are related to the Ubuntu distribution, similar commands are available in other distributions.  
+
==Python examples==
 +
===Connecting to multiple robots===
 +
A simple Python 3 script was developed as a starting point to open a connection with multiple robots and exchange data with them using the [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#WiFi_2 WiFi communication protocol]. The demo was tested with 10 robots but can be easily extended to connect to more robots.<br/>
 +
You can download the script with the command <code>git clone https://github.com/e-puck2/e-puck2_python_wifi_multi.git</code>. The code was tested to work with Python 3.x.
  
==Turn on/off the robot==
+
=Communication protocol=
To turn on the robot you need to press the power button (blue button) placed on the bottom side of the board, near the speaker, as shown in the following figures:
+
This section is the hardest part to understand. It outlines all the details about the communication protocols that you'll need to implement in order to communicate with the robot form the computer. So spend a bit of time reading and re-reading this section in order to grasp completely all the details.
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-btn-on-off2.jpg <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-btn-on-off2-small.jpg">][http://projects.gctronic.com/epuck2/wiki_images/e-puck2-btn-on-off.jpg <img width=300 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-btn-on-off-small.jpg">]</span><br/>
 
To turn off the robot you need to press the power button for 1 second.
 
  
==Meaning of the LEDs==
+
==Bluetooth and USB==
The e-puck2 has three groups of LEDs that are not controllable by the user.
+
The communication protocol is based on the [http://www.gctronic.com/doc/index.php/Advanced_sercom_protocol advanced sercom protocol], used with the e-puck1.x robot. The <code>advanced sercom v2</code> includes all the commands available in the <code>advanced sercom</code> protocol and add some additional commands to handle the new features of the e-puck2 robot. In particular here are the new commands:
 +
{| border="1" cellpadding="10" cellspacing="0"
 +
!Command
 +
!Description
 +
!Return value / set value
 +
|-
 +
|<code>0x08</code>
 +
|Get all sensors
 +
|<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/packet-format-robot-to-pc.jpg <img width=1150 src="http://projects.gctronic.com/epuck2/wiki_images/packet-format-robot-to-pc.jpg">]</span>
 +
see section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#WiFi_2 Communication protocol: WiFi] for the content description
 +
|-
 +
|<code>0x09</code>
 +
|Set all actuators
 +
|<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/packet-format-pc-to-robot-bt.jpg <img width=600 src="http://projects.gctronic.com/epuck2/wiki_images/packet-format-pc-to-robot-bt.jpg">]</span>
 +
see section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#WiFi_2 Communication protocol: WiFi] for the content description
 +
|-
 +
|<code>0x0A</code>
 +
|Set RGB LEDs, values from 0 (off) to 100 (completely on)
 +
|<code>[LED2_red][LED2_green][LED2_blue][LED4_red][LED4_green][LED4_blue][LED6_red][LED6_green][LED6_blue][LED8_red][LED8_green][LED8_blue]</code>
 +
|-
 +
|<code>0x0B</code>
 +
|Get button state: 0 = not pressed, 1 = pressed
 +
|<code>[STATE]</code>
 +
|-
 +
|<code>0x0C</code>
 +
|Get all 4 microphones volumes
 +
|<code>[MIC0_LSB][MIC0_MSB][MIC1_LSB][MIC1_MSB][MIC2_LSB][MIC2_MSB][MIC3_LSB][MIC3_MSB]</code>
 +
|-
 +
|<code>0x0D</code>
 +
|Get distance from ToF sensor (millimeters)
 +
|<code>[DIST_LSB][DIST_MSB]</code>
 +
|-
 +
|<code>0x0E</code>
 +
|Get SD state: 0 = micro sd not connected, 1 = micro sd connected
 +
|<code>[STATE]</code>
 +
|}
  
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2_top_leds.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2_top_leds.png">]</span><br/>
+
==WiFi==
::''Top view of the e-puck2''
+
The communication is based on TCP; the robot create a TCP server and wait for a connection.<br/>
  
*Charger: RED if charging, GREEN if charge complete and RED and GREEN if an error occurs
+
Each packet is identified by an ID (1 byte). The following IDs are used to send data from the robot to the computer:
*USB: Turned ON if the e-puck2 detects a USB connection with a computer
+
* 0x00 = reserved
*STATUS: Turned ON if the robot is ON and OFF if the robot is OFF. When ON, gives an indication of the level of the battery. Also blinks GREEN if the program is running during a debug session.
+
* 0x01 = QQVGA color image packet (only the first segment includes this id); packet size (without id) = 38400 bytes; image format = RGB565
 +
* 0x02 = sensors packet; packet size (without id) = 104 bytes; the format of the returned values are based on the [http://www.gctronic.com/doc/index.php/Advanced_sercom_protocol advanced sercom protocol] and are compatible with e-puck1.x:
  
Battery level indications (STATUS RGB LED):
+
:<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/packet-format-robot-to-pc.jpg <img width=1150 src="http://projects.gctronic.com/epuck2/wiki_images/packet-format-robot-to-pc.jpg">]</span><br/>
*GREEN if the system's tension is greater than 3.5V
+
:*Acc: raw axes values, between -1500 and 1500, resolution is +-2g
*ORANGE if the system's tension is between 3.5V and 3.4V
+
:*Acceleration: acceleration magnitude <img width=70 src="http://projects.gctronic.com/epuck2/wiki_images/3dvector-magnitude.png">, between 0.0 and about 2600.0 (~3.46 g)
*RED if the system's tension is between 3.4V and 3.3V
+
:*Orientation: between 0.0 and 360.0 degrees <table><tr><td align="center">0.0 deg</td><td align="center">90.0 deg</td><td align="center">180 deg</td><td align="center">270 deg</td></tr><tr><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/orientation0.png"></td><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/orientation90.png"></td><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/orientation180.png"></td><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/orientation270.png"></td></tr></table>
*RED blinking if the system's tension is below 3.3V
 
  
The robot is automatically turned OFF if the system's tension gets below 3.2V during 10 seconds.
+
:*Inclination: between 0.0 and 90.0 degrees (when tilted in any direction)<table><tr><td align="center">0.0 deg</td><td align="center">90.0 deg</td></tr><tr><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/inclination0.png"></td><td><img width=80 src="http://projects.gctronic.com/epuck2/wiki_images/inclination90.png"></td></tr></table>
 +
:*Gyro: raw axes values, between -32768 and 32767, range is +-250dps
 +
:*Magnetometer: raw axes values expressed in float, range is +-4912.0 uT (magnetic flux density expressed in micro Tesla)
 +
:*Temp: temperature given in Celsius degrees
 +
:*IR proximity: between 0 (no objects detected) and 4095 (object near the sensor)
 +
:*IR ambient: between 0 (strong light) and 4095 (dark)
 +
:*ToF distance: distance given in millimeters
 +
:*Mic volume: between 0 and 4095
 +
:*Motors steps: 1000 steps per wheel revolution
 +
:*Battery:
 +
:*uSD state: 1 if the micro sd is present and can be read/write, 0 otherwise
 +
:*TV remote data: RC5 protocol
 +
:*Selector position: between 0 and 15
 +
:*Ground proximity: between 0 (no surface at all or not reflective surface e.g. black) and 1023 (very reflective surface e.g. white)
 +
:*Ground ambient: between 0 (strong light) and 1023 (dark)
 +
:*Button state: 1 button pressed, 0 button released
 +
* 0x03 = empty packet (only id is sent); this is used as an acknowledgment for the commands packet when no sensors and no image is requested
 +
The following IDs are used to send data from the computer to the robot:
 +
* 0x80 = commands packet; packet size (without id) = 20 bytes:
  
==Connecting the USB cable==
+
:<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/packet-format-pc-to-robot.jpg <img width=600 src="http://projects.gctronic.com/epuck2/wiki_images/packet-format-pc-to-robot.jpg">]</span><br/>
A micro USB cable (included with the robot in the package) is needed to connect the robot to the computer. There are two connectors, one placed on top of the robot facing upwards and the other placed on the side of the robot, as shown in the following figures. Both can be used to charge the robot (up to 1 ampere) or to communicate with it, but do not connect two cables at the same time. Connect the USB cable where is more comfortable to you.
 
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-usb-conn.jpg <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-usb-conn-small.jpg">][http://projects.gctronic.com/epuck2/wiki_images/e-puck2-usb-conn2.jpg <img width=300 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-usb-conn2-small.jpg">]</span><br/>
 
  
==Installing the USB drivers==
+
:*request:
The USB drivers must be installed only for the users of a Windows version older than Windows 10:
+
:** bit0: 0=stop image stream; 1=start image stream
 +
:** bit1: 0=stop sensors stream; 1=start sensors stream
 +
:*settings:
 +
:** bit0: 1=calibrate IR proximity sensors
 +
:** bit1: 0=disable onboard obstacle avoidance; 1=enable onboard obstacle avoidance (not implemented yet)
 +
:** bit2: 0=set motors speed; 1=set motors steps (position)
 +
:*left and right: when bit2 of <code>settings</code> field is <code>0</code>, then this is the desired motors speed (-1000..1000); when <code>1</code> then this is the value that will be set as motors position (steps)
 +
:*LEDs: 0=off; 1=on
 +
:** bit0: 0=LED1 off; 1=LED1 on
 +
:** bit1: 0=LED3 off; 1=LED3 on
 +
:** bit2: 0=LED5 off; 1=LED5 on
 +
:** bit3: 0=LED7 off; 1=LED7 on
 +
:** bit4: 0=body LED off; 1=body LED on
 +
:** bit5: 0=front LED off; 1=front LED on
 +
:*RGB LEDs: for each LED, it is specified in sequence the value of red, green and blue (0...100)
 +
:* sound id: 0x01=MARIO, 0x02=UNDERWOLRD, 0x04=STARWARS, 0x08=4KHz, 0x10=10KHz, 0x20=stop sound
  
#Download and open [http://projects.gctronic.com/epuck2/zadig-2.3.exe zadig-2.3.exe]
+
For example to receive the camera image (stream) the following steps need to be followed:<br/>
#Connect the e-puck2 with the USB cable and turn it on. Three unknown devices appear in the device list of the program, namely '''e-puck2 STM32F407''', '''e-puck2 GDB Server (Interface 0)''' and '''e-puck2 Serial Monitor (Interface 2)'''.
+
1) connect to the robot through TCP<br/>
#For each of the three devices mentioned above, select the <code>USB Serial (CDC)</code> driver and click on the <code>Install Driver</code> button to install it. Accept the different prompts which may appear during the process. At the end you can simply quit the program and the drivers are installed. These steps are illustrated on Figure 3 below.
+
2) send the command packet:
::Note : The drivers installed are located in <code>C:\Users\"your_user_name"\usb_driver</code>
+
:{| border="1"  
 +
|0x80
 +
|0x01
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|0x00
 +
|}
 +
3) read the ID (1 byte) and the QQVGA color image pakcet (38400 bytes)<br/>
 +
4) go to step 3
  
:<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/Zadig_e-puck2_STM32F407.png <img width=500 src="http://projects.gctronic.com/epuck2/wiki_images/Zadig_e-puck2_STM32F407.png">]</span><br/>
+
=Webots=
::''Example of driver installation for e-puck2 STM32F407''
+
TBD
  
The drivers are automatically installed with Windows 10, Linux and Mac OS.
+
=ROS=
 +
This chapter explains how to use ROS with the e-puck2 robots by connecting them via Bluetooth or WiFi to the computer that runs the ROS nodes. Basically all the sensors are exposed to ROS and you can also send commands back to the robot through ROS. Both Pyhton and cpp versions are implemented to give the user the possibility to choose its preferred programming language. Here is a general schema:<br/>
 +
<span class="plainlinks">[http://www.gctronic.com/doc/images/epuck2-ros-schema.png <img width=450 src="http://www.gctronic.com/doc/images/epuck2-ros-schema-small.png">]</span>
 +
''<font size="2">Click to enlarge</font>''<br/>
  
Anyway in Linux in order to access the serial ports, a little configuration is needed. Type the following command in a terminal session: <code>sudo adduser $USER dialout</code>. Once done, you need to log off to let the change take effect.
+
First of all you need to install and configure ROS, refer to [http://wiki.ros.org/Distributions http://wiki.ros.org/Distributions] for more informations. <font style="color:red"> This tutorial is based on ROS Kinetic</font>. The same instructions are working with ROS Noetic, beware to use <code>noetic</code> instead of <code>kinetic</code> when installing the packages.
  
==Finding the USB serial ports used==
+
Starting from the work done with the e-puck1 (see [https://www.gctronic.com/doc/index.php?title=E-Puck#ROS E-Puck ROS]), we updated the code in order to support the e-puck2 robot.
Two ports are created by the e-puck2's programmer when the USB cable is connected to the robot (even if the robot is turned off):
 
* '''e-puck2 GDB Server'''. The port used to program and debug the e-puck2.
 
* '''e-puck2 Serial Monitor'''. Serial communication between the PC and the radio module (used also to program the radio module).
 
  
A third port could be available depending on the code inside the e-puck2's microcontroller. With the factory firmware a port named '''e-puck2 STM32F407''' is created.
+
==Initial configuration==
===Windows===
+
The following steps need to be done only once, after installing ROS:
#Open the Device Manager
+
:1. If not already done, create a catkin workspace, refer to [http://wiki.ros.org/catkin/Tutorials/create_a_workspace http://wiki.ros.org/catkin/Tutorials/create_a_workspace]. Basically you need to issue the following commands: 
#Under '''Ports (COM & LPT)''' you can see the virtual ports connected to your computer.
+
<pre>  mkdir -p ~/catkin_ws/src
#Do a '''Right-click -> properties''' on the COM port you want to identify.
+
  cd ~/catkin_ws/src
#Go under the '''details''' tab and select '''Bus reported device description''' in the properties list.
+
  catkin_init_workspace
#The name of the port should be written in the text box below.
+
  cd ~/catkin_ws/
#Once you found the desired device, you can simply look at its port number '''(COMX)'''.
+
  catkin_make
 +
  source devel/setup.bash </pre>
 +
:2. You will need to add the line <code>source ~/catkin_ws/devel/setup.bash</code> to your <tt>.bashrc</tt> in order to automatically have access to the ROS commands when the system is started
 +
:3. Move to <code>~/catkin_ws/src</code> and clone the ROS e-puck2 driver repo:
 +
:* if you are working with Python (only Bluetooth communication supported at the moment): <code>git clone -b e-puck2 https://github.com/gctronic/epuck_driver</code>
 +
:* if you are working with cpp:
 +
:** Bluetooth communication: <code>git clone -b e-puck2 https://github.com/gctronic/epuck_driver_cpp</code>
 +
:** WiFi communication: <code>git clone -b e-puck2_wifi https://github.com/gctronic/epuck_driver_cpp</code>
 +
:4. Install the dependencies:
 +
:* ROS:
 +
:** [http://wiki.ros.org/gmapping gmapping (SLAM)] package: <code>sudo apt-get install ros-kinetic-gmapping</code>
 +
:** [http://wiki.ros.org/rviz_imu_plugin Rviz IMU plugin] package: <code>sudo apt-get install ros-kinetic-rviz-imu-plugin</code>
 +
:* Python:
 +
:** The ROS e-puck2 driver is based on the e-puck2 Python library that requires some dependencies:
 +
:*** install the Python setup tools: <code>sudo apt-get install python-setuptools</code>
 +
:*** install the Python image library: <code>sudo apt-get install python-imaging</code>
 +
:*** install pybluez version 0.22: <code>sudo pip install pybluez==0.22</code>
 +
:**** install pybluez dependencies: <code>sudo apt-get install libbluetooth-dev</code>
 +
:*** install OpenCV: <code>sudo apt-get install python3-opencv</code>
 +
:* cpp:
 +
:** install the library used to communicate with Bluetooth: <code>sudo apt-get install libbluetooth-dev</code>
 +
:** install OpenCV: <code>sudo apt-get install libopencv-dev</code>
 +
:*** if you are working with OpenCV 4, then you need to change the header include from <code>#include <opencv/cv.h></code> to <code>#include <opencv2/opencv.hpp></code>
 +
:5. Open a terminal and go to the catkin workspace directory (<tt>~/catkin_ws</tt>) and issue the command <code>catkin_make</code>, there shouldn't be errors
 +
:6. Program the e-puck2 robot with the [https://www.gctronic.com/doc/index.php?title=e-puck2#Factory_firmware factory firmware] and put the selector in position 3 for Bluetooth communication or in position 15 for WiFi Communication
 +
:7. Program the radio module with the correct firmware:
 +
:* Bluetooth communication: use the [https://www.gctronic.com/doc/index.php?title=e-puck2#Factory_firmware_2 factory firmware]
 +
:* WiFi communication: use the [https://www.gctronic.com/doc/index.php?title=e-puck2#WiFi_firmware WiFi firmware]
  
===Linux===
+
==Running the Python ROS node==
:1. Open a terminal window (<code>ctrl+alt+t</code>) and enter the following command: <code>ls /dev/ttyACM*</code>
+
First of all get the last version of the ROS e-puck2 driver from github. Move to <code>~/catkin_ws/src</code> and issue: <code>git clone -b e-puck2 https://github.com/gctronic/epuck_driver</code>. <br/>
:2. Look for '''ttyACM0''' and '''ttyACM1''' in the generated list, which are respectively '''e-puck2 GDB Server''' and '''e-puck2 Serial Monitor'''. '''ttyACM2''' will be also available with the factory firmware, that is related to '''e-puck2 STM32F407''' port
+
Then build the driver by opening a terminal and issueing the command <code>catkin_make</code> from within the catkin workspace directory (e.g. ~/catkin_ws).<br/>
Note : Virtual serial port numbering on Linux depends on the connections order, thus it can be different if another device using virtual serial ports is already connected to your computer before connecting the robot, but the sequence remains the same.
+
Moreover make sure the node is marked as executable by opening a terminal and issueing the following command from within the catkin workspace directory (e.g. ~/catkin_ws): <code>chmod +x ./src/epuck_driver/scripts/epuck2_driver.py</code>. <br/>
  
===Mac===
+
Before actually starting the e-puck2 node you need to configure the e-puck2 robot as Bluetooth device in the system, refer to section [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_Bluetooth Connecting to the Bluetooth].<br/>
:1. Open a terminal window and enter the following command: <code>ls /dev/cu.usbmodem*</code>
+
Once the robot is paired with the computer, you need to take note of its MAC address (this will be needed when launching the ROS node). To know the MAC address of a paired robot, go to <tt>System Settings</tt>, <tt>Bluetooth</tt> and select the robot; once selected you'll see in the right side the related MAC address.
:2. Look for two '''cu.usbmodemXXXX''', where XXXX is the number attributed by your computer. You should find two names, with a numbering near to each other, which are respectively '''e-puck2 GDB Server''' (lower number) and '''e-puck2 Serial Monitor''' (higher number). A third device '''cu.usbmodemXXXX''' will be available with the factory firmware, that is related to '''e-puck2 STM32F407''' port
 
  
Note : Virtual serial port numbering on Mac depends on the physical USB port used and the device. If you want to keep the same names, you must connect to the same USB port each time.
+
First thing to do before launching the script file is running the <tt>roscore</tt>, open another terminal tab and issue the command <tt>roscore</tt>.
  
==PC interface==
+
Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on [http://wiki.ros.org/roslaunch roslaunch]).<br/>
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/monitor.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/monitor_small.png">]<br/>
+
Open a terminal and issue the following command: <code>roslaunch epuck_driver epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F'</code>.<br/>
A PC application was developed to start playing with the robot attached to the computer via USB cable: you can have information about all the sensors, receive camera images and control the leds and motors.<br/>
+
<tt>B4:E6:2D:EB:9C:4F</tt> is the e-puck2 Bluetooth MAC address that need to be changed accordingly to your robot.
Beware that it's not mandatory to download this application in order to work with the robot, but it is a nice demo that gives you an overview of all the sensors and actuators available on the robot, this is a first step to gain confidence with the robot.<br/>
 
  
With the factory firmwares programmed in the robot, place the selector in position 8, attach the USB cable and turn on the robot. Enter the correct port (the one related to <code>e-puck2 STM32F407</code>) in the interface and click <code>connect</code>.
+
If all is going well you'll see the robot make a blink meaning it is connected and ready to exchange data and [http://wiki.ros.org/rviz/UserGuide rviz] will be opened showing the informations gathered from the topics published by the e-puck2 driver node.
  
The source code is available from the repository [https://github.com/e-puck2/monitor https://github.com/e-puck2/monitor].<br/>
+
The launch script is configured also to run the [http://wiki.ros.org/gmapping gmapping (SLAM)] node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.
  
===Available executables===
+
The following figures show all the topics published by the e-puck2 driver node (left) and the <code>rviz</code> interface (right): <br/>
* [http://projects.gctronic.com/epuck2/monitor_win.zip Windows executable]: tested on Windows 7 and Windows 10
+
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics.png <img width=200 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics_small.png">]</span>
* [http://projects.gctronic.com/epuck2/monitor_mac.zip Max OS X executable]
+
''<font size="2">Click to enlarge</font>''
* [http://projects.gctronic.com/epuck2/monitor_linux64bit.tar.gz Ubuntu 14.04 (or later) - 64 bit]
+
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2-rviz.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2-rviz_small.png">]</span>
On Linux remember to apply the configuration explained in the chapter [http://www.gctronic.com/doc/index.php?title=e-puck2#Installing_the_USB_drivers Installing the USB drivers] in order to access the serial port.
+
''<font size="2">Click to enlarge</font>''<br/>
  
==Installing the dependencies for firmwares updates==
+
==Running the cpp ROS node==
You can update the firmware for all 3 chips: the main microcontroller, the radio module and the programmer. For doing that, you need some tools to be installed on the system.
+
There is a small difference at the moment between the Bluetooth and WiFi versions of the ROS node: the WiFi ROS node supports also the publication of the magnetometer data.
 +
===Bluetooth===
 +
First of all get the last version of the ROS e-puck2 driver from github. Move to <code>~/catkin_ws/src</code> and issue: <code>git clone -b e-puck2 https://github.com/gctronic/epuck_driver_cpp</code>. <br/>
 +
Then build the driver by opening a terminal and issueing the command <code>catkin_make</code> from within the catkin workspace directory (e.g. ~/catkin_ws).<br/>
  
===Windows===
+
Before actually starting the e-puck2 node you need to configure the e-puck2 robot as Bluetooth device in the system, refer to section [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_Bluetooth Connecting to the Bluetooth].<br/>
To upload a new firmware in the microcontroller or in the radio module, you don't need to install anything, the packages provided include all the dependencies.
+
Once the robot is paired with the computer, you need to take note of its MAC address (this will be needed when launching the ROS node). To know the MAC address of a paired robot, go to <tt>System Settings</tt>, <tt>Bluetooth</tt> and select the robot; once selected you'll see in the right side the related MAC address.
  
To upload a new firmware in the programmer you need to install an application called <code>DfuSe</code> released by STMicroelectronics. You can download it from [http://projects.gctronic.com/epuck2/en.stsw-stm32080_DfuSe_Demo_V3.0.5.zip DfuSe_V3.0.5.zip].
+
First thing to do before launching the script file is running the <tt>roscore</tt>, open another terminal tab and issue the command <tt>roscore</tt>.
  
===Linux===
+
Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on [http://wiki.ros.org/roslaunch roslaunch]).<br/>
To upload a new firmware in the microcontroller or in the radio module, you need:
+
Open a terminal and issue the following command: <code>roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F'</code>.<br/>
* Python (>= 3.4): <code>sudo apt-get install python3</code>
+
<tt>B4:E6:2D:EB:9C:4F</tt> is the e-puck2 Bluetooth MAC address that need to be changed accordingly to your robot.
* Python pip: <code>sudo apt-get install -y python3-pip</code>
 
* pySerial (>= 2.5): <code>sudo pip3 install pyserial</code>
 
  
To upload a new firmware in the programmer you need:
+
If all is going well the robot will be ready to exchange data and [http://wiki.ros.org/rviz/UserGuide rviz] will be opened showing the informations gathered from the topics published by the e-puck2 driver node.
* dfu-util: <code>sudo apt-get install dfu-util</code>
 
  
===Mac===
+
The launch script is configured also to run the [http://wiki.ros.org/gmapping gmapping (SLAM)] node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.
Install the [https://brew.sh Homewbrew] package manager by opening a terminal and issueing:<br/>
+
===WiFi===
<code>/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"</code><br/>
+
First of all get the last version of the ROS e-puck2 driver from github. Move to <code>~/catkin_ws/src</code> and issue: <code>git clone -b e-puck2_wifi https://github.com/gctronic/epuck_driver_cpp</code>. <br/>
and then:<br/>
+
Then build the driver by opening a terminal and issueing the command <code>catkin_make</code> from within the catkin workspace directory (e.g. ~/catkin_ws).<br/>
<code>brew upgrade</code><br/>
 
  
To upload a new firmware in the microcontroller or in the radio module, you need:
+
Before actually starting the e-puck2 node you need to connect the e-puck2 robot to your WiFi network, refer to section [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_WiFi Connecting to the WiFi].<br/>
* Python (>= 3.4): <code>brew install python</code> (it will install also <code>pip</code>)
 
* pySerial (>= 2.5): <code>pip3 install pyserial</code>  
 
  
To upload a new firmware in the programmer you need:
+
First thing to do before launching the script file is running the <tt>roscore</tt>, open another terminal tab and issue the command <tt>roscore</tt>.
* dfu-util: <code>brew install dfu-util</code>
 
  
==PC side development==
+
Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on [http://wiki.ros.org/roslaunch roslaunch]).<br/>
This section is dedicated to the users that develop algorithms on the PC side and interact with the robot remotely through a predefined communication protocol. These users don't modify the firmware of the robot, but instead they use the factory firmware released with the robot. They update the robot firmware only when there is an official update. <br/>
+
Open a terminal and issue the following command: <code>roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='192.168.1.20'</code>.<br/>
The remote control of the robot, by receiving sensors values and setting the actuators, is done through the following channels: Bluetooth, Bluetooth Low Energy, WiFi, USB cable.<br/>
+
<tt>192.168.1.20</tt> is the e-puck2 IP address that need to be changed accordingly to your robot.
Examples of tools/environment used by these users:
 
# Aseba
 
# Simulator (e.g. Webots)
 
# ROS
 
# iOS, Android apps
 
# Custom PC application
 
# IoT (e.g. IFTTT)
 
If you fall into this category, then follow this section for more information: [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development PC side development].<br/>
 
  
=Main microcontroller=
+
If all is going well the robot will be ready to exchange data and [http://wiki.ros.org/rviz/UserGuide rviz] will be opened showing the informations gathered from the topics published by the e-puck2 driver node.
The e-puck2 robot main microcontroller is a 32-bit STM32F407 that runs at 168 MHz (210 DMIPS) and include DSP, FPU and DMA capabilities. The version chosen for the e-puck2 has 192 KB of total RAM and 1024 KB of flash, so there is a lot of memory to work with.<br/>
 
This chip is responsible for handling the sensors and actuators and runs also the demos and algorithms.
 
  
==Factory firmware==
+
The launch script is configured also to run the [http://wiki.ros.org/gmapping gmapping (SLAM)] node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.
The main microcontroller of the robot is initially programmed with a firmware that includes many demos that could be started based on the selector position, here is a list of the demos with related position and a small description:
 
* Selector position 0: Aseba
 
* Selector position 1: Shell
 
* Selector position 2: Read proximity sensors and when an object is near a proximity, turn on the corresponding LED
 
* Selector position 3: Asercom protocol v2 (BT)
 
* Selector positoin 4: Range and bearing extension (receiver)
 
* Selector position 5: Range and bearing extension (transmitter)
 
* Selector position 6: Move the robot back and forth exploiting the gyro, with LEDs animation
 
* Selector position 7: Play a wav (mono, 16 KHz) named "example.wav" from the micro sd when pressing the button
 
* Selector position 8: Asercom protocol v2 (USB)
 
* Selector position 9: Local communication: transceiver
 
* Selector position 10: this position is used to work with the Linux extensions. To work with gumstix refer to [http://www.gctronic.com/doc/index.php?title=Overo_Extension#e-puck2 Overo Extension: e-puck2] , to work with Pi-puck refer to [http://www.gctronic.com/doc/index.php?title=Pi-puck#Requirements Pi-puck: Requirements ].
 
* Selector position 11: Simple obstacle avoidance + some animation
 
* Selector position 12: Hardware test
 
* Selector position 13: LEDs reflect orientation of the robot
 
* Selector position 14: Compass
 
* Selector position 15: WiFi mode
 
The pre-built firmware is available here [http://projects.gctronic.com/epuck2/e-puck2_main-processor_16.04.20_a83c0a1.elf main microcontroller factory firmware (16.04.20)].
 
  
==Firmware update==
+
The refresh rate of the topics is about 11 Hz when the camera image is enabled (see [http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics_wifi_refresh_camon.pdf e-puck2_topics_wifi_refresh_camon.pdf]) and about 50 Hz when the camera image is disabled (see [http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics_wifi_refresh_camoff.pdf e-puck2_topics_wifi_refresh_camoff.pdf]). The same graphs can be created using the command <code>rosrun tf view_frames</code>.
Now and then there could be an official firmware update for the robot and it's important to keep the robot updated with the last firmware to get possibile new features, improvements and for bug fixes.<br/>
 
The onboard programmer run a GDB server, so we use GDB commands to upload a new firmware, for this reason a toolchain is needed to upload a new firmware to the robot.<br/>
 
The following steps explain how to update the main microcontroller firmware:<br/>
 
1. Download the package containing the required toolchain and script to program the robot: [http://projects.gctronic.com/epuck2/e-puck2-prog-main-micro-windows.zip Windows], Linux [http://projects.gctronic.com/epuck2/e-puck2-prog-main-linux32.tar.gz 32 bits]/[http://projects.gctronic.com/epuck2/e-puck2-prog-main-linux64.tar.gz 64 bits], [http://projects.gctronic.com/epuck2/e-puck2-prog-main-micro-macos.zip Mac OS]<br/>
 
2. Download the last version of the [http://projects.gctronic.com/epuck2/e-puck2_main-processor_16.04.20_a83c0a1.elf main microcontroller factory firmware (16.04.20)], or use your custom firmware<br/>
 
3. Extract the package and put the firmware file (with <code>elf</code> extension) inside the package directory; beware that only one <code>elf</code> file must be present inside this directory<br/>
 
4. Attach the USB cable and turn on the robot<br/>
 
5. Run the script from the package directory:<br/>
 
:Windows: double click <code>program.bat</code><br/>
 
:Linux/Mac: issue the following command in a terminal <code>./program.sh</code>. If you get permission errors, then issue <code>sudo chmod +x program.sh</code> to let the script be executable.<br/>
 
  
When the upload is complete you'll see an output like in the following figure:<br/>
+
The following figure shows all the topics published by the e-puck2 WiFi ROS node. The same graph can be created using the command <code>rqt_graph</code>. <br/>
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/f407-flashing.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/f407-flashing.png">]</span><br/>
+
<span class="plainlinks">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics_wifi.png <img width=200 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2_topics_wifi.png">]</span>
The final lines should contain the entry <code>".data",</code>, this means that the upload was successfull. You can then close the terminal window if it is still open.
+
''<font size="2">Click to enlarge</font>''
  
If you encounter some problem, try to unplug and plug again the USB cable and power cycle the robot, then retry.
+
==Move the robot==
 +
You have some options to move the robot.<br/>
  
==Robot side development==
+
The first one is to use the <code>rviz</code> interface: in the bottom left side of the interface there is a <code>Teleop</code> panel containing an ''interactive square'' meant to be used with differential drive robots. By clicking in this square you'll move the robot, for instance by clicking on the top-right section, then the robot will move forward-right.<br/>
If you are an embedded developer and are brave enough, then you have complete access to the source code running on the robot, so you can discover what happen inside the main microcontroller and modify it to accomodate your needs. Normally the users that fall into this category develop algorithms optimized to run directly on the microcontroller, such as:
 
# onboard image processing
 
# swarm algorithms
 
# fully autonomous behaviors
 
# ...
 
For more information about programming the robot itself, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2_robot_side_development Robot side development]
 
  
=Radio module=
+
The second method to move the robot is using the <code>ros-kinetic-turtlebot-teleop</code> ROS package. If not already done, you can install this package by issueing <code>sudo apt-get install ros-kinetic-turtlebot-teleop</code>.<br/>
The radio module chosen for the e-puck is the new ESP32 chip from [https://www.espressif.com/ Espressif], integrating a dual core that run up to 240 MHz, 4 MB of flash and 520 KB of RAM. It supports WiFi standards 802.11 b/g/n (access point mode supported), Bluetooth and Bluetooth LE 4.2. It is the successor of the ESP8266 chip. The following figure shows the various peripherals available on the ESP32:<br/>
+
There is a lunch file in the e-puck2 ROS driver that configures this package in order to be used with the e-puck2 robot. To start the launch file, issue the following command <code>roslaunch epuck_driver epuck2_teleop.launch</code>, then follow the instructions printed on the terminal to move the robot.<br/>
<span class="plain links"><img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-peripherals.png"></span>
 
  
This chip first of all is responsible for handling the wireless communication, moreover it handles also the RGB LEDs (with PWM) and the user button. The RGB LEDs and button are connected to the radio module due to the pin number limitation on the main microcontroller.
+
The third method is by directly publishing on the <code>/mobile_base/cmd_vel</code> topic, for instance by issueing the following command <code>rostopic pub -1 /mobile_base/cmd_vel geometry_msgs/Twist -- '[0.0, 0.0, 0.0]' '[0.0, 0.0, 1.0]'</code> the robot will rotate on the spot, instead by issueing the following command <code>rostopic pub -1 /mobile_base/cmd_vel geometry_msgs/Twist -- '[4.0, 0.0, 0.0]' '[0.0, 0.0, 0.0]'</code> the robot will move straight forward.<br/>
 +
Beware that there shouldn't be any other node publishing on the <code>/mobile_base/cmd_vel</code> topic, otherwise your commands will be overwritten.
  
==Factory firmware==
+
==Control the RGB LEDs==
The radio module of the robot is initially programmed with a firmware that supports Bluetooth communication.<br/>
+
The general command to change the RGB LEDs colors is the following:<br/>
The pre-built firmware is available here [http://projects.gctronic.com/epuck2/esp32-firmware_11.12.18.zip radio module factory firmware (11.12.18)].
+
<code>rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [LED2 red, LED2 green, LED2 blue, LED4 red, LED4 green, LED4 blue, LED6 red, LED6 green, LED6 blue, LED8 red, LED8 green, LED8 blue]}"</code><br/>
 +
The values range is from 0 (off) to 100 (completely on). Have a look at the [https://www.gctronic.com/doc/index.php?title=e-puck2#Overview e-puck2 overview] to know the position of the RGB LEDs.<br/>
  
==WiFi firmware==
+
For instance to set all the RGB LEDs to red, issue the following command:<br/>
At the moment the factory firmware supports only Bluetooth, if you want to work with WiFi you need to program the radio with a dedicated firmware, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#WiFi PC side development: WiFi].
+
<code>rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [100,0,0, 100,0,0, 100,0,0, 100,0,0]}"</code><br/>
  
==BLE firmware==
+
To turn off all the RGB LEDs issue the following command:<br/>
At the moment the factory firmware supports only calssic Bluetooth, if you want to work with Bluetooth Low Energy you need to program the radio with a dedicated firmware, refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2_mobile_phone_development Mobile phone development].
+
<code>rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [0,0,0, 0,0,0, 0,0,0, 0,0,0]}"</code>
  
==Firmware update==
+
==Control the LEDs==
In order to update the firmware of the ESP32 WiFi module you need to use a python script called <code>esptool</code> provided by [https://www.espressif.com/ Espressif] (manufacturer of the chip). This script was modified to work with the e-puck2 robot and is included in the provided package. The following steps explain how to update the radio module firmware:<br/>
+
The general command to change the LEDs state is the following:<br/>
1. Download the package containing the required tools and script to program the robot: [http://projects.gctronic.com/epuck2/e-puck2-prog-radio-windows.zip Windows], [http://projects.gctronic.com/epuck2/e-puck2-prog-radio-macos.zip Linux / Mac]<br/>
+
<code>rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [LED1, LED3, LED5, LED7, body LED, front LED]}"</code><br/>
2. Download the last version of the [http://projects.gctronic.com/epuck2/esp32-firmware_11.12.18.zip radio module factory firmware (11.12.18)], or use another firmware (e.g. WiFi, BLE, your own). The firmware is composed by 3 files named <code>bootloader.bin</code>, <code>ESP32_E-Puck_2.bin</code> and <code>partitions_singleapp.bin</code><br/>
+
The values are: 0 (off), 1 (on) and 2 (toggle). Have a look at the [https://www.gctronic.com/doc/index.php?title=e-puck2#Overview e-puck2 overview] to know the position of the LEDs.<br/>
3. Extract the package and put the firmware files inside the package directory; beware that the name of the <code>.bin</code> files must be the same as indicated in step 2<br/>
 
4. Attach the USB cable and turn on the robot<br/>
 
5. Run the script from the package directory:<br/>
 
:Windows: double click <code>program.bat</code><br/>
 
:Linux/Mac: issue the following command in a terminal <code>./program.sh</code>. If you get permission errors, then issue <code>sudo chmod +x program.sh</code> to let the script be executable.<br/>
 
  
The upload should last about 10-15 seconds and you'll see the progress as shown in the following figure:<br/>
+
For instance to turn on LED1, LED5, body LED and front LED, issue the following command:<br/>
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing1.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing1.png">]</span><br/>
+
<code>rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [1,0,1,0,1,1]}"</code><br/>
When the upload is complete you'll see that all 3 bin files are uploaded correctly as shown in the following figure:<br/>
 
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing2.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing2.png">]</span><br/>
 
  
Sometime you could encounter a timeout error as shown in the following figures; in these cases you need to unplug and plug again the USB cable and power cycle the robot, then you can retry.<br/>
+
To toggle the state of all the LEDs issue the following command:<br/>
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing3.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing3.png">]</span>
+
<code>rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [2,2,2,2,2,2]}"</code>
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing4.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/esp32-flashing4.png">]</span><br/>
 
  
==Development==
+
==Visualize the camera image==
Probably, you'll never need to touch the firmware running in the radio module, but in case you need to modify the code or you're simply curious about what is happening at the low level, then refer to the section [http://www.gctronic.com/doc/index.php?title=e-puck2_radio_module_development Radio module development].
+
By default the camera is disabled to avoid communication delays. In order to enable it and visualize the image through ROS you need to pass an additional parameter <code>cam_en</code> to the launch script as follows:<br/>
 +
* Python: <code>roslaunch epuck_driver epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F' cam_en:='true'</code>
 +
* cpp:
 +
** Bluetooth: <code>roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F' cam_en:='true'</code>
 +
** WiFi: <code>roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='192.168.1.20' cam_en:='true'</code>
  
=Programmer=
+
Then with the Python ROS node you need to open another terminal and issue the command <code>rosrun image_view image_view image:=/camera</code> that will open a window with the e-puck2 camera image.<br/>
The e-puck2 robot is equipped with an onboard programmer and debugger that let you update the firmware of the robot and debug your code easily using a standard USB interface. There is a dedicated STM32F413 microcontroller that acts as the programmer with built in GDB server, so you can control exactly what happens using the [https://www.gnu.org/software/gdb/ GNU Project Debugger] in your host machine.<br/>
+
With the cpp ROS node the image is visualized directly in the Rviz window (on the right).<br/>
The programmer microcontroller is also in charge of handling various low level features such as the configuration of the USB hub and the power button.
 
  
==Factory firmware==
+
When using the Bluetooth ROS node, by default the image is greyscale and its size is 160x2, but you can change the image parameters in the launch script.<br/>
The programmer is initially programmed with a firmware based on a '''''modified version''''' of [https://github.com/blacksphere/blackmagic/wiki Black Magic Probe programmer/debugger].<br/>
+
Instead when using the WiFi node, the image is RGB565 and its size is fixed to 160x120 (you can't change it).
The pre-built firmware is available here [http://projects.gctronic.com/epuck2/e-puck2_programmer_28.05.20_3b600ec.bin programmer-firmware.bin (28.05.20)]; it is also available in dfu format here [http://projects.gctronic.com/epuck2/e-puck2_programmer_28.05.20_3b600ec.dfu programmer-firmware.dfu (28.05.20)].
+
==Multiple robots==
 +
There is a lunch script file designed to run up to 4 robots simultaneously, you can find it in <code>~/catkin_ws/src/epuck_driver_cpp/launch/multi_epuck2.launch</code>. Here is an example to run 2 robots:<br/>
 +
<code>roslaunch epuck_driver_cpp multi_epuck2.launch robot_addr0:='192.168.1.21' robot_addr1:='192.168.1.23'</code><br/>
 +
After issueing the command, rviz will be opened showing the values of all the 4 robots; it is assumed that the robots are placed in a square (each robot in each corner) of 20 cm.
  
==Firmware update==
+
==Troubleshooting==
The programmer's microcontroller features a factory bootloader that can be entered by acting on some special pins, the bootloader mode is called DFU (device firmware upgrade). You can enter DFU mode by contacting two pinholes together while inserting the USB cable (no need to turn on the robot). The two pin holes are located near the USB connector of the e-puck2, see the photo below.
+
===Robot state publisher===
 +
If you get an error similar to the following when you start a node with roslaunch:
 +
<pre>
 +
ERROR: cannot launch node of type [robot_state_publisher/state_publisher]: Cannot locate node of type [state_publisher] in package [robot_state_publisher]. Make sure file exists in package path and permission is set to executable (chmod +x)
 +
</pre>
 +
Then you need to change the launch file from:
 +
<pre>
 +
<node name="robot_state_publisher" pkg="robot_state_publisher" type="state_publisher" />
 +
</pre>
 +
To:
 +
<pre>
 +
<node name="robot_state_publisher" pkg="robot_state_publisher" type="robot_state_publisher" />
 +
</pre>
 +
This is due to the fact that <code>state_publisher</code> was a deprecated alias for the node named <code>robot_state_publisher</code> (see [https://github.com/ros/robot_state_publisher/pull/87 https://github.com/ros/robot_state_publisher/pull/87]).
  
::<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/e-puck2_top_leds_DFU_413.png <img width=200 src="http://projects.gctronic.com/epuck2/wiki_images/e-puck2_top_leds_DFU_413.png">]</span><br/>
+
=Tracking=
::''Location of the pin holes to put the programmer into DFU''
+
Some experiments are done with the [https://en.wikibooks.org/wiki/SwisTrack SwisTrack software] in order to be able to track the e-puck2 robots through a color marker placed on top of the robots.
  
The programmer will be recognized as <code>STM Device in DFU Mode</code> device.
+
The requirements are the following:
 +
* e-puck robots equipped with a color marker attached on top of the robot; beware that there should be a white border of about 1 cm to avoid wrong detection (marker merging). The colors marker were printed with a laser printer.
 +
* USB webcam with a resolution of at least 640x480. In our tests we used the <code>Trust SpotLight Pro</code>.
 +
* Windows OS: the SwisTrack pre-compiled package was built to run in Windows. Moreover the controller example depends on Windows libraries.<br/>''Anyway it's important to notice that SwisTrack is multiplatform and that the controller code can be ported to Linux.
 +
* An arena with uniform light conditions to make the detection more robust.
  
'''Note for Windows users''': the device should be recognized automatically (in all Windows versions), but in case it won't be detected then you need to install a <code>libusbK</code> driver for the DFU device.<br>
+
==Controller example==
Follow the same procedure as explained in section [http://www.gctronic.com/doc/index.php?title=e-puck2#Installing_the_USB_drivers Installing the USB drivers] using <code>libusbK</code> driver instead of <code>USB Serial (CDC)</code>.
+
In this example we exploit the ''SwisTrack'' blobs detection feature in order to detect the color markers on top of the robots and then track these blob with a ''Nearest Neighbour tracking'' algorithm.<br/>
 +
The ''SwisTrack'' application get an image from the USB camera, then applies some conversions and thresholding before applying the blobs detection and finally tracks these blobs. All the data, like the blob's positions, are published to the network (TCP). <br/>
 +
The controller is a separate application that receives the data from SwisTrack through the network and opens a Bluetooth connection with each robot in order to remote control them. In the example, the informations received are printed in the terminal while moving the robots around (obstacles avoidance).<br/>
 +
The following schema shows the connections schema:<br/>
 +
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/tracking-schema.png <img width=400 src="http://projects.gctronic.com/epuck2/wiki_images/tracking-schema.png">]</span><br/>
  
===Linux/Mac===
 
In order to update the programmer firmware you need an utility called <code>dfu-util</code>, it should be already installed from section [http://www.gctronic.com/doc/index.php?title=e-puck2#Installing_the_dependencies_for_firmwares_updates Installing the dependencies for firmwares updates].<br/>
 
To uplaod the firmware, issue the following command: <code>sudo dfu-util -d 0483:df11 -a 0 -s 0x08000000 -D programmer-firmware.bin</code>
 
  
===Windows===
+
Follow these steps to run the example:
Start the <code>DfuSe</code> application (previously installed from section [http://www.gctronic.com/doc/index.php?title=e-puck2#Installing_the_dependencies_for_firmwares_updates Installing the dependencies for firmwares updates]). The programmer in DFU mode will be automatically detected as shown in figure 1. Then you need to open the compiled firmware by clicking on <code>choose</code> and then locating the file with <code>dfu</code> extension,  as shown in figure 2. Now click on the <code>upgrade</code> button, a warning message will be shown, confirm the action by clicking on <code>yes</code> as shown in figure 3. If all is ok you'll be prompted with a message saying that the upgrade was successfull as shown in figure 4.<br/>
+
* program all the e-puck2 robots with the last factory firmware (see section [https://www.gctronic.com/doc/index.php?title=e-puck2#Firmware_update Firmware update]) and put the selector in position 3
<span class="plainlinks">
+
* pair the robots with the computer, refer to section [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#Connecting_to_the_Bluetooth Connecting to the Bluetooth]
<table>
+
* the controller example is based on the [https://www.gctronic.com/doc/index.php?title=e-puck2_PC_side_development#C.2B.2B_remote_library C++ remote library], so download it
<tr>
+
* download the controller example by issueing the following command: <code>git clone https://github.com/e-puck2/e-puck2_tracking_example</code>.<br/> When building the example, make sure that both the library and the example are in the same directory
<td align="center">[1]</td>
+
* download the pre-compiled [http://projects.gctronic.com/elisa3/SwisTrackEnvironment-10.04.13.zip SwisTrack software] and extract it. The ''SwisTrack'' executable can be found in <code>SwisTrackEnvironment/SwisTrack - Release.exe</code>
<td align="center">[2]</td>
+
* prepare the arena: place the USB camera on the roof pointing towards the robots. Download the [http://projects.gctronic.com/epuck2/tracking/e-puck2-tracking-markers.pdf markers] and attach one of them on top of each robot.
<td align="center">[3]</td>
+
* download the [http://projects.gctronic.com/epuck2/tracking/swistrack-conf.zip configuration files package] for ''SwisTrack'' and extract it. Run the ''SwisTrack'' executable and open the configuration file called <code>epuck2.swistrack</code>. All the components to accomplish the tracking of '''2 robots''' should be loaded automatically.<br/> If needed you can tune the various components to improve the blobs detection in your environment or for tracking more robots.
<td align="center">[4]</td>
+
* Run the controller example: at the beginning you must enter the Bluetooth UART port numbers for the 2 robots. Then the robots will be moved slightly in order to identify which robot belong to which blob. Then the controller loop is started sending motion commands to the robots for doing obstacles avoidance and printing the data received from SwisTrack in the terminal.
</tr>
 
<tr>
 
<td>[http://projects.gctronic.com/epuck2/wiki_images/dfu1.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/dfu1.png">]</td>
 
<td>[http://projects.gctronic.com/epuck2/wiki_images/dfu2.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/dfu2.png">]</td>
 
<td>[http://projects.gctronic.com/epuck2/wiki_images/dfu3.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/dfu3.png">]</td>
 
<td>[http://projects.gctronic.com/epuck2/wiki_images/dfu4.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/dfu4.png">]</td>
 
</tr>
 
</table>
 
</span><br/>
 
  
==Development==
+
The following image shows the example running:<br/>
The programmer code shouldn't be modified, but if you know what you're doing then refer to section [http://www.gctronic.com/doc/index.php?title=e-puck2_programmer_development Programmer development].
+
<span class="plain links">[http://projects.gctronic.com/epuck2/wiki_images/tracking-epuck2.png <img width=250 src="http://projects.gctronic.com/epuck2/wiki_images/tracking-epuck2_small.png">]</span><br/>

Revision as of 12:25, 18 November 2020

e-puck2 main wiki

1 Robot configuration

This section explains how to configure the robot based on the communication channel you will use for your developments, thus you need to read only one of the following sections, but it would be better if you spend a bit of time reading them all in order to have a full understanding of the available configurations.

1.1 USB

The main microcontroller is initially programmed with a firmware that support USB communication.

If the main microcontroller isn't programmed with the factory firmware or if you want to be sure to have the last firmware on the robot, you need to program it with the last factory firmware by referring to section main microcontroller firmware update.

The radio module can be programmed with either the Bluetooth or the WiFi firmware, both are compatible with USB communication:

When you want to interact with the robot from the computer you need to place the selector in position 8 to work with USB.

Section PC interface gives step by step instructions on how to connect the robot with the computer via USB.

Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both USB and Bluetooth communication channels use the same protocol called advanced sercom v2, refer to section Communication protocol: BT and USB for detailed information about this protocol.

1.2 Bluetooth

The main microcontroller and radio module of the robot are initially programmed with firmwares that together support Bluetooth communication.

If the main microcontroller and radio module aren't programmed with the factory firmware or if you want to be sure to have the last firmwares on the robot, you need to program them with the last factory firmwares:

When you want to interact with the robot from the computer you need to place the selector in position 3 if you want to work with Bluetooth.

Section Connecting to the Bluetooth gives step by step instructions on how to accomplish your first Bluetooth connection with the robot.

Once you tested the connection with the robot and the computer, you can start developing your own application by looking at the details behind the communication protocol. Both Bluetooth and USB communication channels use the same protocol called advanced sercom v2, refer to section Communication protocol: BT and USB for detailed information about this protocol.

1.3 WiFi

For working with the WiFi, the main microcontroller must be programmed with the factory firmware and the radio module must be programmed with a dedicated firmware (not the factory one):

Put the selector in position 15.

Section Connecting to the WiFi gives step by step instructions on how to accomplish your first WiFi connection with the robot.

The communication protocol is described in detail in the section Communication protocol: WiFi.

2 Connecting to the Bluetooth

The factory firmware of the radio module creates 3 Bluetooth channels using the RFcomm protocol when the robot is paired with the computer:

  1. Channel 1, GDB: port to connect with GDB if the programmer is in mode 1 or 3 (refer to chapter Configuring the Programmer's settings for more information about these modes)
  2. Channel 2, UART: port to connect to the UART port of the main processor
  3. Channel 3, SPI: port to connect to the SPI port of the main processor (not yet implemented. Just do an echo for now)

By default, the e-puck2 is not visible when you search for it in the Bluetooth utility of your computer.
To make it visible, it is necessary to hold the USER button (also labeled "esp32" on the electronic board) while turning on the robot with the ON/OFF button.


Then it will be discoverable and you will be able to pair with it.
Note that a prompt could ask you to confirm that the number written on the screen is the same on the e-puck. just ignore this and accept. Otherwise if you are asked for a pin insert 0000.

2.1 Windows 7

When you pair your computer with the e-puck2, 3 COM ports will be automatically created. To see which COM port corresponds to which channel you need to open the properties of the paired e-puck2 robot from Bluetooth devices. Then the ports and related channels are listed in the Services tab, as shown in the following figure:

2.2 Windows 10

When you pair your computer with the e-puck2, 6 COM ports will be automatically created. The three ports you will use have Outgoing direction and are named e_puck2_xxxxx-GDB, e_puck2_xxxxx-UART, e_puck2_xxxxx-SPI. xxxxx is the ID number of your e-puck2.
To see which COM port corresponds to which channel you need to:

  1. open the Bluetooth devices manager
  2. pair with the robot
  3. click on More Bluetooth options
  4. the ports and related channels are listed in the COM Ports tab, as shown in the following figure:

2.3 Linux

Once paired with the Bluetooth manager, you need to create the port for communicating with the robot by issueing the command:
sudo rfcomm bind /dev/rfcomm0 MAC_ADDR 2
The MAC address is visible from the Bluetooth manager. The parameter 2 indicates the channel, in this case a port for the UART channel is created. If you want to connect to another service you need to change this parameter accordingly (e.g. 1 for GDB and 3 for SPI). Now you can use /dev/rfcomm0 to connect to the robot.

2.4 Mac

When you pair your computer with the e-puck2, 3 COM ports will be automatically created: /dev/cu.e-puck2_xxxxx-GDB, /dev/cu.e-puck2_xxxxx-UART and /dev/cu.e-puck2_xxxxx-SPI. xxxxx is the ID number of your e-puck2.

2.5 Testing the Bluetooth connection

You need to download the PC application provided in section PC interface: available executables.
In the connection textfield you need to enter the UART channel port, for example:

  • Windows 7: COM258
  • Windows 10: e_puck2_xxxxx-UART
  • Linux: /dev/rfcomm0
  • Mac: /dev/cu.e-puck2_xxxxx-UART

and then click Connect.
You should start receiving sensors data and you can send commands to the robot.

Alternatively you can also use a simple terminal program (e.g. realterm in Windows) instead of the PC application, then you can issue manually the commands to receive sensors data or for setting the actuators (once connected, type h + ENTER for a list of availables commands).

2.6 Python examples

Here are some basic Python examples that show how to get data from the robot through Bluetooth using the commands available with the advanced sercom v2:

In all the examples you need to set the correct Bluetooth serial port related to the robot.

2.6.1 Connecting to multiple robots

Here is a simple Python script multi-robot.py that open a connection with 2 robots and exchange data with them using the advanced sercom protocol. This example can be extended to connect to more than 2 robots.

2.7 C++ remote library

A remote control library implemented in C++ is available to control the e-puck2 robot via a Bluetooth connection from the computer.
The remote control library is multiplatform and uses only standard C++ libraries.
You can download the library with the command git clone https://github.com/e-puck2/e-puck2_cpp_remote_library.
A simple example showing how to use the library is also available; you can download it with the command git clone https://github.com/e-puck2/e-puck2_cpp_remote_example.
Before building the example you need to build the library. Then when building the example, make sure that both the library and the example are in the same directory, that is you must end up with the following directory tree:

e-puck2_projects
|_ e-puck2_cpp_remote_library
|_ e-puck2_cpp_remote_example

The complete API reference is available in the following link e-puck2_cpp_remote_library_api_reference.pdf.

3 Connecting to the WiFi

The WiFi channel is used to communicate with robot faster than with Bluetooth. At the moment a QQVGA (160x120) color image is transferred to the computer together with the sensors values at about 10 Hz; of course the robot is also able to receive commands from the computer.
In order to communicate with the robot through WiFi, first you need to configure the network parameters on the robot by connecting directly to it, since the robot is initially configured in access point mode, as explained in the following section. Once the configuration is saved on the robot, it will then connect automatically to the network and you can connect to it.

The LED2 is used to indicate the state of the WiFi connection:

  • red indicates that the robot is in access point mode (waiting for configuration)
  • green indicates that the robot is connected to a network and has received an IP address
  • blue (toggling) indicates that the robot is transferring the image to the computer
  • off when the robot cannot connect to the saved configuration

3.1 Network configuration

If there is no WiFi configuration saved in flash, then the robot will be in access point mode in order to let the user connect to it and setup a WiFi connection. The LED2 is red.

The access point SSID will be e-puck2_0XXXX where XXXX is the id of the robot; the password to connect to the access point is e-puck2robot.
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address 192.168.1.1. The available networks are scanned automatically and listed in the browser page as shown in figure 1. Choose the WiFi signal you want the robot to establish a conection with from the web generated list, and enter the related password; if the password is correct you'll get a message saying that the connection is established as shown in figure 2. After pressing OK you will be redirected to the main page showing the network to which you're connected and the others available nearby as shown in figure 3. If you press on the connected network, then you can see your IP address as shown in figure 4; take note of the address since it will be needed later.

[1] [2] [3] [4]


Now the configuration is saved in flash, this means that when the robot is turned on it will read this configuration and try to establish a connection automatically.
Remember that you need to power cycle the robot at least once for the new configuration to be active.

Once the connection is established, the LED2 will be green.

In order to reset the current configuration you need to press the user button for 2 seconds (the LED2 red will turn on), then you need to power cycle the robot to enter access point mode.


3.2 Finding the IP address

Often the IP address assigned to the robot will remain the same when connecting to the same network, so if you took note of the IP address in section Network configuration you're ready to go to the next section.

Otherwise you need to connect the robot to the computer with the USB cable, open a terminal and connect to the port labeled Serial Monitor (see chapter Finding the USB serial ports used). Then power cycle the robot and the IP address will be shown in the terminal (together with others informations), as illustrated in the following figure:

3.3 Testing the WiFi connection

A dedicated WiFi version of the PC application was developed to communicate with the robot through TCP protocol. You can download the executable from one of the following links:

If you are interested to the source code, you can download it with the command git clone -b wifi --recursive https://github.com/e-puck2/monitor.git

Run the PC application, insert the IP address of the robot in the connection textfield and then click on the Connect button. You should start receiving sensors data and you can send commands to the robot. The LED2 blue will toggle.

3.4 Web server

When the robot is in access point mode you can have access to a web page showing the camera image and some buttons that you can use to move the robot; it is a basic example that you can use as a starting point to develop your own web browser interface.
You can use a phone, a tablet or a computer to connect to the robot's WiFi and then you need to open a browser and insert the address 192.168.1.1/monitor.html.

3.5 Python examples

3.5.1 Connecting to multiple robots

A simple Python 3 script was developed as a starting point to open a connection with multiple robots and exchange data with them using the WiFi communication protocol. The demo was tested with 10 robots but can be easily extended to connect to more robots.
You can download the script with the command git clone https://github.com/e-puck2/e-puck2_python_wifi_multi.git. The code was tested to work with Python 3.x.

4 Communication protocol

This section is the hardest part to understand. It outlines all the details about the communication protocols that you'll need to implement in order to communicate with the robot form the computer. So spend a bit of time reading and re-reading this section in order to grasp completely all the details.

4.1 Bluetooth and USB

The communication protocol is based on the advanced sercom protocol, used with the e-puck1.x robot. The advanced sercom v2 includes all the commands available in the advanced sercom protocol and add some additional commands to handle the new features of the e-puck2 robot. In particular here are the new commands:

Command Description Return value / set value
0x08 Get all sensors

see section Communication protocol: WiFi for the content description

0x09 Set all actuators

see section Communication protocol: WiFi for the content description

0x0A Set RGB LEDs, values from 0 (off) to 100 (completely on) [LED2_red][LED2_green][LED2_blue][LED4_red][LED4_green][LED4_blue][LED6_red][LED6_green][LED6_blue][LED8_red][LED8_green][LED8_blue]
0x0B Get button state: 0 = not pressed, 1 = pressed [STATE]
0x0C Get all 4 microphones volumes [MIC0_LSB][MIC0_MSB][MIC1_LSB][MIC1_MSB][MIC2_LSB][MIC2_MSB][MIC3_LSB][MIC3_MSB]
0x0D Get distance from ToF sensor (millimeters) [DIST_LSB][DIST_MSB]
0x0E Get SD state: 0 = micro sd not connected, 1 = micro sd connected [STATE]

4.2 WiFi

The communication is based on TCP; the robot create a TCP server and wait for a connection.

Each packet is identified by an ID (1 byte). The following IDs are used to send data from the robot to the computer:

  • 0x00 = reserved
  • 0x01 = QQVGA color image packet (only the first segment includes this id); packet size (without id) = 38400 bytes; image format = RGB565
  • 0x02 = sensors packet; packet size (without id) = 104 bytes; the format of the returned values are based on the advanced sercom protocol and are compatible with e-puck1.x:

  • Acc: raw axes values, between -1500 and 1500, resolution is +-2g
  • Acceleration: acceleration magnitude , between 0.0 and about 2600.0 (~3.46 g)
  • Orientation: between 0.0 and 360.0 degrees
    0.0 deg90.0 deg180 deg270 deg
  • Inclination: between 0.0 and 90.0 degrees (when tilted in any direction)
    0.0 deg90.0 deg
  • Gyro: raw axes values, between -32768 and 32767, range is +-250dps
  • Magnetometer: raw axes values expressed in float, range is +-4912.0 uT (magnetic flux density expressed in micro Tesla)
  • Temp: temperature given in Celsius degrees
  • IR proximity: between 0 (no objects detected) and 4095 (object near the sensor)
  • IR ambient: between 0 (strong light) and 4095 (dark)
  • ToF distance: distance given in millimeters
  • Mic volume: between 0 and 4095
  • Motors steps: 1000 steps per wheel revolution
  • Battery:
  • uSD state: 1 if the micro sd is present and can be read/write, 0 otherwise
  • TV remote data: RC5 protocol
  • Selector position: between 0 and 15
  • Ground proximity: between 0 (no surface at all or not reflective surface e.g. black) and 1023 (very reflective surface e.g. white)
  • Ground ambient: between 0 (strong light) and 1023 (dark)
  • Button state: 1 button pressed, 0 button released
  • 0x03 = empty packet (only id is sent); this is used as an acknowledgment for the commands packet when no sensors and no image is requested

The following IDs are used to send data from the computer to the robot:

  • 0x80 = commands packet; packet size (without id) = 20 bytes:

  • request:
    • bit0: 0=stop image stream; 1=start image stream
    • bit1: 0=stop sensors stream; 1=start sensors stream
  • settings:
    • bit0: 1=calibrate IR proximity sensors
    • bit1: 0=disable onboard obstacle avoidance; 1=enable onboard obstacle avoidance (not implemented yet)
    • bit2: 0=set motors speed; 1=set motors steps (position)
  • left and right: when bit2 of settings field is 0, then this is the desired motors speed (-1000..1000); when 1 then this is the value that will be set as motors position (steps)
  • LEDs: 0=off; 1=on
    • bit0: 0=LED1 off; 1=LED1 on
    • bit1: 0=LED3 off; 1=LED3 on
    • bit2: 0=LED5 off; 1=LED5 on
    • bit3: 0=LED7 off; 1=LED7 on
    • bit4: 0=body LED off; 1=body LED on
    • bit5: 0=front LED off; 1=front LED on
  • RGB LEDs: for each LED, it is specified in sequence the value of red, green and blue (0...100)
  • sound id: 0x01=MARIO, 0x02=UNDERWOLRD, 0x04=STARWARS, 0x08=4KHz, 0x10=10KHz, 0x20=stop sound

For example to receive the camera image (stream) the following steps need to be followed:
1) connect to the robot through TCP
2) send the command packet:

0x80 0x01 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

3) read the ID (1 byte) and the QQVGA color image pakcet (38400 bytes)
4) go to step 3

5 Webots

TBD

6 ROS

This chapter explains how to use ROS with the e-puck2 robots by connecting them via Bluetooth or WiFi to the computer that runs the ROS nodes. Basically all the sensors are exposed to ROS and you can also send commands back to the robot through ROS. Both Pyhton and cpp versions are implemented to give the user the possibility to choose its preferred programming language. Here is a general schema:
Click to enlarge

First of all you need to install and configure ROS, refer to http://wiki.ros.org/Distributions for more informations. This tutorial is based on ROS Kinetic. The same instructions are working with ROS Noetic, beware to use noetic instead of kinetic when installing the packages.

Starting from the work done with the e-puck1 (see E-Puck ROS), we updated the code in order to support the e-puck2 robot.

6.1 Initial configuration

The following steps need to be done only once, after installing ROS:

1. If not already done, create a catkin workspace, refer to http://wiki.ros.org/catkin/Tutorials/create_a_workspace. Basically you need to issue the following commands:
  mkdir -p ~/catkin_ws/src
  cd ~/catkin_ws/src
  catkin_init_workspace
  cd ~/catkin_ws/
  catkin_make
  source devel/setup.bash 
2. You will need to add the line source ~/catkin_ws/devel/setup.bash to your .bashrc in order to automatically have access to the ROS commands when the system is started
3. Move to ~/catkin_ws/src and clone the ROS e-puck2 driver repo:
4. Install the dependencies:
  • ROS:
  • Python:
    • The ROS e-puck2 driver is based on the e-puck2 Python library that requires some dependencies:
      • install the Python setup tools: sudo apt-get install python-setuptools
      • install the Python image library: sudo apt-get install python-imaging
      • install pybluez version 0.22: sudo pip install pybluez==0.22
        • install pybluez dependencies: sudo apt-get install libbluetooth-dev
      • install OpenCV: sudo apt-get install python3-opencv
  • cpp:
    • install the library used to communicate with Bluetooth: sudo apt-get install libbluetooth-dev
    • install OpenCV: sudo apt-get install libopencv-dev
      • if you are working with OpenCV 4, then you need to change the header include from #include <opencv/cv.h> to #include <opencv2/opencv.hpp>
5. Open a terminal and go to the catkin workspace directory (~/catkin_ws) and issue the command catkin_make, there shouldn't be errors
6. Program the e-puck2 robot with the factory firmware and put the selector in position 3 for Bluetooth communication or in position 15 for WiFi Communication
7. Program the radio module with the correct firmware:

6.2 Running the Python ROS node

First of all get the last version of the ROS e-puck2 driver from github. Move to ~/catkin_ws/src and issue: git clone -b e-puck2 https://github.com/gctronic/epuck_driver.
Then build the driver by opening a terminal and issueing the command catkin_make from within the catkin workspace directory (e.g. ~/catkin_ws).
Moreover make sure the node is marked as executable by opening a terminal and issueing the following command from within the catkin workspace directory (e.g. ~/catkin_ws): chmod +x ./src/epuck_driver/scripts/epuck2_driver.py.

Before actually starting the e-puck2 node you need to configure the e-puck2 robot as Bluetooth device in the system, refer to section Connecting to the Bluetooth.
Once the robot is paired with the computer, you need to take note of its MAC address (this will be needed when launching the ROS node). To know the MAC address of a paired robot, go to System Settings, Bluetooth and select the robot; once selected you'll see in the right side the related MAC address.

First thing to do before launching the script file is running the roscore, open another terminal tab and issue the command roscore.

Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on roslaunch).
Open a terminal and issue the following command: roslaunch epuck_driver epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F'.
B4:E6:2D:EB:9C:4F is the e-puck2 Bluetooth MAC address that need to be changed accordingly to your robot.

If all is going well you'll see the robot make a blink meaning it is connected and ready to exchange data and rviz will be opened showing the informations gathered from the topics published by the e-puck2 driver node.

The launch script is configured also to run the gmapping (SLAM) node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.

The following figures show all the topics published by the e-puck2 driver node (left) and the rviz interface (right):
Click to enlarge Click to enlarge

6.3 Running the cpp ROS node

There is a small difference at the moment between the Bluetooth and WiFi versions of the ROS node: the WiFi ROS node supports also the publication of the magnetometer data.

6.3.1 Bluetooth

First of all get the last version of the ROS e-puck2 driver from github. Move to ~/catkin_ws/src and issue: git clone -b e-puck2 https://github.com/gctronic/epuck_driver_cpp.
Then build the driver by opening a terminal and issueing the command catkin_make from within the catkin workspace directory (e.g. ~/catkin_ws).

Before actually starting the e-puck2 node you need to configure the e-puck2 robot as Bluetooth device in the system, refer to section Connecting to the Bluetooth.
Once the robot is paired with the computer, you need to take note of its MAC address (this will be needed when launching the ROS node). To know the MAC address of a paired robot, go to System Settings, Bluetooth and select the robot; once selected you'll see in the right side the related MAC address.

First thing to do before launching the script file is running the roscore, open another terminal tab and issue the command roscore.

Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on roslaunch).
Open a terminal and issue the following command: roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F'.
B4:E6:2D:EB:9C:4F is the e-puck2 Bluetooth MAC address that need to be changed accordingly to your robot.

If all is going well the robot will be ready to exchange data and rviz will be opened showing the informations gathered from the topics published by the e-puck2 driver node.

The launch script is configured also to run the gmapping (SLAM) node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.

6.3.2 WiFi

First of all get the last version of the ROS e-puck2 driver from github. Move to ~/catkin_ws/src and issue: git clone -b e-puck2_wifi https://github.com/gctronic/epuck_driver_cpp.
Then build the driver by opening a terminal and issueing the command catkin_make from within the catkin workspace directory (e.g. ~/catkin_ws).

Before actually starting the e-puck2 node you need to connect the e-puck2 robot to your WiFi network, refer to section Connecting to the WiFi.

First thing to do before launching the script file is running the roscore, open another terminal tab and issue the command roscore.

Now you can finally start the e-puck2 ROS node, for this purposes there is a launch script (based on roslaunch).
Open a terminal and issue the following command: roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='192.168.1.20'.
192.168.1.20 is the e-puck2 IP address that need to be changed accordingly to your robot.

If all is going well the robot will be ready to exchange data and rviz will be opened showing the informations gathered from the topics published by the e-puck2 driver node.

The launch script is configured also to run the gmapping (SLAM) node that let the robot construct a map of the environment; the map is visualized in real-time directly in the rviz window. The gmapping package provides laser-based SLAM (Simultaneous Localization and Mapping) and since the e-puck2 has no laser sensor, the information from the 6 proximity sensors on the front side of the robot are interpolated to get 19 laser scan points.

The refresh rate of the topics is about 11 Hz when the camera image is enabled (see e-puck2_topics_wifi_refresh_camon.pdf) and about 50 Hz when the camera image is disabled (see e-puck2_topics_wifi_refresh_camoff.pdf). The same graphs can be created using the command rosrun tf view_frames.

The following figure shows all the topics published by the e-puck2 WiFi ROS node. The same graph can be created using the command rqt_graph.
Click to enlarge

6.4 Move the robot

You have some options to move the robot.

The first one is to use the rviz interface: in the bottom left side of the interface there is a Teleop panel containing an interactive square meant to be used with differential drive robots. By clicking in this square you'll move the robot, for instance by clicking on the top-right section, then the robot will move forward-right.

The second method to move the robot is using the ros-kinetic-turtlebot-teleop ROS package. If not already done, you can install this package by issueing sudo apt-get install ros-kinetic-turtlebot-teleop.
There is a lunch file in the e-puck2 ROS driver that configures this package in order to be used with the e-puck2 robot. To start the launch file, issue the following command roslaunch epuck_driver epuck2_teleop.launch, then follow the instructions printed on the terminal to move the robot.

The third method is by directly publishing on the /mobile_base/cmd_vel topic, for instance by issueing the following command rostopic pub -1 /mobile_base/cmd_vel geometry_msgs/Twist -- '[0.0, 0.0, 0.0]' '[0.0, 0.0, 1.0]' the robot will rotate on the spot, instead by issueing the following command rostopic pub -1 /mobile_base/cmd_vel geometry_msgs/Twist -- '[4.0, 0.0, 0.0]' '[0.0, 0.0, 0.0]' the robot will move straight forward.
Beware that there shouldn't be any other node publishing on the /mobile_base/cmd_vel topic, otherwise your commands will be overwritten.

6.5 Control the RGB LEDs

The general command to change the RGB LEDs colors is the following:
rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [LED2 red, LED2 green, LED2 blue, LED4 red, LED4 green, LED4 blue, LED6 red, LED6 green, LED6 blue, LED8 red, LED8 green, LED8 blue]}"
The values range is from 0 (off) to 100 (completely on). Have a look at the e-puck2 overview to know the position of the RGB LEDs.

For instance to set all the RGB LEDs to red, issue the following command:
rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [100,0,0, 100,0,0, 100,0,0, 100,0,0]}"

To turn off all the RGB LEDs issue the following command:
rostopic pub -1 /mobile_base/rgb_leds std_msgs/UInt8MultiArray "{data: [0,0,0, 0,0,0, 0,0,0, 0,0,0]}"

6.6 Control the LEDs

The general command to change the LEDs state is the following:
rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [LED1, LED3, LED5, LED7, body LED, front LED]}"
The values are: 0 (off), 1 (on) and 2 (toggle). Have a look at the e-puck2 overview to know the position of the LEDs.

For instance to turn on LED1, LED5, body LED and front LED, issue the following command:
rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [1,0,1,0,1,1]}"

To toggle the state of all the LEDs issue the following command:
rostopic pub -1 /mobile_base/cmd_led std_msgs/UInt8MultiArray "{data: [2,2,2,2,2,2]}"

6.7 Visualize the camera image

By default the camera is disabled to avoid communication delays. In order to enable it and visualize the image through ROS you need to pass an additional parameter cam_en to the launch script as follows:

  • Python: roslaunch epuck_driver epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F' cam_en:='true'
  • cpp:
    • Bluetooth: roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='B4:E6:2D:EB:9C:4F' cam_en:='true'
    • WiFi: roslaunch epuck_driver_cpp epuck2_controller.launch epuck2_address:='192.168.1.20' cam_en:='true'

Then with the Python ROS node you need to open another terminal and issue the command rosrun image_view image_view image:=/camera that will open a window with the e-puck2 camera image.
With the cpp ROS node the image is visualized directly in the Rviz window (on the right).

When using the Bluetooth ROS node, by default the image is greyscale and its size is 160x2, but you can change the image parameters in the launch script.
Instead when using the WiFi node, the image is RGB565 and its size is fixed to 160x120 (you can't change it).

6.8 Multiple robots

There is a lunch script file designed to run up to 4 robots simultaneously, you can find it in ~/catkin_ws/src/epuck_driver_cpp/launch/multi_epuck2.launch. Here is an example to run 2 robots:
roslaunch epuck_driver_cpp multi_epuck2.launch robot_addr0:='192.168.1.21' robot_addr1:='192.168.1.23'
After issueing the command, rviz will be opened showing the values of all the 4 robots; it is assumed that the robots are placed in a square (each robot in each corner) of 20 cm.

6.9 Troubleshooting

6.9.1 Robot state publisher

If you get an error similar to the following when you start a node with roslaunch:

ERROR: cannot launch node of type [robot_state_publisher/state_publisher]: Cannot locate node of type [state_publisher] in package [robot_state_publisher]. Make sure file exists in package path and permission is set to executable (chmod +x)

Then you need to change the launch file from:

<node name="robot_state_publisher" pkg="robot_state_publisher" type="state_publisher" />

To:

<node name="robot_state_publisher" pkg="robot_state_publisher" type="robot_state_publisher" />

This is due to the fact that state_publisher was a deprecated alias for the node named robot_state_publisher (see https://github.com/ros/robot_state_publisher/pull/87).

7 Tracking

Some experiments are done with the SwisTrack software in order to be able to track the e-puck2 robots through a color marker placed on top of the robots.

The requirements are the following:

  • e-puck robots equipped with a color marker attached on top of the robot; beware that there should be a white border of about 1 cm to avoid wrong detection (marker merging). The colors marker were printed with a laser printer.
  • USB webcam with a resolution of at least 640x480. In our tests we used the Trust SpotLight Pro.
  • Windows OS: the SwisTrack pre-compiled package was built to run in Windows. Moreover the controller example depends on Windows libraries.
    Anyway it's important to notice that SwisTrack is multiplatform and that the controller code can be ported to Linux.
  • An arena with uniform light conditions to make the detection more robust.

7.1 Controller example

In this example we exploit the SwisTrack blobs detection feature in order to detect the color markers on top of the robots and then track these blob with a Nearest Neighbour tracking algorithm.
The SwisTrack application get an image from the USB camera, then applies some conversions and thresholding before applying the blobs detection and finally tracks these blobs. All the data, like the blob's positions, are published to the network (TCP).
The controller is a separate application that receives the data from SwisTrack through the network and opens a Bluetooth connection with each robot in order to remote control them. In the example, the informations received are printed in the terminal while moving the robots around (obstacles avoidance).
The following schema shows the connections schema:


Follow these steps to run the example:

  • program all the e-puck2 robots with the last factory firmware (see section Firmware update) and put the selector in position 3
  • pair the robots with the computer, refer to section Connecting to the Bluetooth
  • the controller example is based on the C++ remote library, so download it
  • download the controller example by issueing the following command: git clone https://github.com/e-puck2/e-puck2_tracking_example.
    When building the example, make sure that both the library and the example are in the same directory
  • download the pre-compiled SwisTrack software and extract it. The SwisTrack executable can be found in SwisTrackEnvironment/SwisTrack - Release.exe
  • prepare the arena: place the USB camera on the roof pointing towards the robots. Download the markers and attach one of them on top of each robot.
  • download the configuration files package for SwisTrack and extract it. Run the SwisTrack executable and open the configuration file called epuck2.swistrack. All the components to accomplish the tracking of 2 robots should be loaded automatically.
    If needed you can tune the various components to improve the blobs detection in your environment or for tracking more robots.
  • Run the controller example: at the beginning you must enter the Bluetooth UART port numbers for the 2 robots. Then the robots will be moved slightly in order to identify which robot belong to which blob. Then the controller loop is started sending motion commands to the robots for doing obstacles avoidance and printing the data received from SwisTrack in the terminal.

The following image shows the example running: