2018年12月5日 星期三

Use STM8 to Control nRF24L01


※The complete code is available in here.

In my previous post, I used nRF24LE1, which is an 8051 microcontroller(MCU) integrated nRF24L01 function, to operate Enhanced Shockburst™ (ESB) transceiving. If your MCU would tackle only some simple tack with wireless function, nRF24LE1 is a good selection. However, in some case, nRF24LE1 may be not a choice. For example, nRF24LE1 computing power is limited and does not support CAN bus, pins of nRF24LE1 would be conflicted for your application, multi-channels ADC/I2C/SPI are required,  or just existing product must not be totally reforged..etc. Hence, In here, I would to show how to use STM8L to manipulate nRF24L01.

The STM8L I use here is STM8L101F3P3 features its low price (STM8L10X series is lowest product line of STM8 series.). Because the STM101F1  flash size is not adequate to containing the firmware operating nRF24L01, I use the chip postfix being -F3 rather than -F1.


 To detail explain how to deal with the porting, I start from my another post code of folder nRF24LE1, which has been completely implemented ESB send and receive functions.

 
 零.
    Download ST Visual develop IDESTM8 Cosmic compiler, and  STM8L10x standard peripheral library. you need to provide your email to STMicroelectronics. Then install the STM8 Cosmic Compiler Crack to break through the 32KB limitation.

Set ST Visual develop using the cosmic  compiler : Tools → Option :



一. Download my the code for nRF24LE1 :

Download the code from here, you should extract those :
esb_app.c
esb_app.h
hal\nordic_common.h
hal\nrf24l01p\hal_nrf.c
hal\nrf24l01p\hal_nrf.h
hal\nrf24l01p\hal_nrf_reg.h
hal\nrf24le1\hal_nrf_hw.c
hal\nrf24le1\hal_nrf_hw.h

compiler\c51\stdin.h


And create a folder to contain those code, the folder structure is :

esb_app.c
esb_app.h
nRF24L01_hal\nordic_common.h
nRF24L01_hal\nrf24l01p\hal_nrf.c
nRF24L01_hal\nrf24l01p\hal_nrf.h
nRF24L01_hal\nrf24l01p\hal_nrf_reg.h
nRF24L01_hal\stm8L10x\stdin.h
nRF24L01_hal\stm8L10x\hal_nrf_hw.c
nRF24L01_hal\stm8L10x\hal_nrf_hw.h


二 .  Modify the necessary code:
The firmware architecture diagram is as below, The blocks framed purple is the code you need to modify/implement.





esb_app.h :
export esb_irq function :


:

void esb_receiving_event_has_been_handled(void);


/*nRF24L01 IRQ*/
#if !defined(NRF24LE1_H_) && !defined(NRF24LU1P_H)
void esb_irq(void);
#endif

esb_app.c :

comment out the line to set register RFCKEN and RF, and remove the keyword "static"  of function esb_irq.

void esb_init(void)
:
#if defined(NRF24LE1_H_) || defined(NRF24LU1P_H)
 RFCKEN = 1;  //enable RF timer
#endif
:
#if defined(NRF24LE1_H_) || defined(NRF24LU1P_H)
 RF = 1; //enable rf interrupt
#endif
:
}/*enhanced_shockburst_init*/

:

/*static */ void esb_irq(void)
{
:


As the diagram shows, the hal_nrf,c and  hal_nrf.h  would be intact, but  hal_nrf_hw.c and hal_nrf_hw.h need to be completely implemented. Is is because the SPI operations are high MCU-dependency, and pin definition is high board-dependency.

The schematic of my board is :


keypoints:

PC0 pin is for nRF24L01 IRQ.
PB4 pin is for nRF24L01 CSN.
PB3 pin is for nRF24L01 CE.

In this manner,  hal_nrf_hw.h as :


#ifndef HAL_NRF_STM8L101XX_H__
#define HAL_NRF_STM8L101XX_H__
#include "STM8l10x_conf.h"

#define data     
#define pdata
#define xdata

/** Macro that set radio's CSN line LOW.
 *
 */
#define CSN_LOW() do {  GPIO_ResetBits( GPIOB, GPIO_Pin_4);} while(false)
/** Macro that set radio's CSN line HIGH.
 *
 */
#define CSN_HIGH() do { GPIO_SetBits( GPIOB, GPIO_Pin_4);} while(false)

/** Macro that set radio's CE line LOW.
 *
 */
#define CE_LOW() do {GPIO_ResetBits(GPIOB, GPIO_Pin_3);} while(false)

/** Macro that set radio's CE line HIGH.
 *
 */
#define CE_HIGH() do {GPIO_SetBits(GPIOB, GPIO_Pin_3);} while(false)

/** Macro for writing the radio SPI data register.
 *
 */
#define HAL_NRF_HW_SPI_WRITE(d) do{\
                 while (SPI_GetFlagStatus(SPI_FLAG_TXE) == RESET);\
                 SPI_SendData(d);\
                } while(false)

/** Macro for reading the radio SPI data register.
 *
 */
#define HAL_NRF_HW_SPI_READ()  SPI_ReceiveData()
  
/** Macro specifyng the radio SPI busy flag.
 *
 */
#define HAL_NRF_HW_SPI_BUSY   (RESET == SPI_GetFlagStatus(SPI_FLAG_RXNE))

/**
 * Pulses the CE to nRF24L01 for at least 10 us
 */

#define CE_PULSE() do { \
  uint8_t count; \
  count = 20; \
  CE_HIGH();  \
  while(count--){} \
  CE_LOW();  \
  } while(false)

 
#endif // HAL_NRF_STM8L101XX_H__

/** @} */


Those lines to define data, pdata and xdata are for avoiding compilation error.

hal_nrf_hw.c:


#include <stdint.h>
#include "hal_nrf.h"
#include "STM8l10x_conf.h"

uint8_t hal_nrf_rw(uint8_t value)
{
 while (RESET == SPI_GetFlagStatus(SPI_FLAG_TXE));
 SPI_SendData(value);
 
 //wait for receiving a byte
 while(RESET == SPI_GetFlagStatus(SPI_FLAG_RXNE));
 return SPI_ReceiveData();
}


三. Create the project.

  Create a workspace of ST Visual Develop with what the chip you use, and create a project inside it. the project folder in this structure:


stm8_interrupt_vector.c and stm8l10x_conf.h are auto-generated files. The  STM8L10x_StdPeriph_Driver folder contains the driver files of of STM8L10x standard peripheral library, in directory STM8L10x_StdPeriph_Lib\Libraries\STM8L10x_StdPeriph_Driver, for inc and src folder respectively.



The next stuff I need to do,  is to complete the nRF24L01 initialization. Pay attention :: it is not ESB initializtion function, which has been implemented in esb_app.c. The nRF24L01 initialization is for initializing the interface resources used in communicated with nRF24L01.

There are two part: initialize the Serial Peripheral Interface Bus(SPI), and the General-purpose input/output(GPIO).

The pins:

PB7 pin is SPI_MISO, input, high = 1.
PB6 pin is SPI_MOSI, output, high = 1.
PB5 pin is SPI_SCK, output, high = 1.

PB4 pin is for nRF24L01 CSN. output, set = 1.
PB3 pin is for nRF24L01 CE, output, set =1.

PC0 pin is for nRF24L01 IRQ, input,  external interrupt, active-low.

According to above definition, the the data sheet of nRF24L01 for SPI timing explanation(CPOL= low, CPHA = rising edge, more detail explanation about CPOL and CPHA you could refer to here) , the code for nRF24L01 initialization is :


void spi_init(void)
{
 //Config the GPIOs for SPI bus
 
 /*
  GPIOB::GPIO_Pin_7  -> SPI_MISO   
  GPIOB::GPIO_Pin_6  -> SPI_MOSI  
  GPIOB::GPIO_Pin_5  -> SPI_SCK   
 */
 GPIO_Init( GPIOB, GPIO_Pin_7, GPIO_Mode_In_PU_No_IT);
 GPIO_Init( GPIOB, GPIO_Pin_5 | GPIO_Pin_6, 
  GPIO_Mode_Out_PP_High_Slow);
  
 SPI_DeInit();
 //enable clock for SPI bus
 CLK_PeripheralClockConfig(CLK_Peripheral_SPI, ENABLE);

/*SPI BAUDRATE should not be over 5MHz*/
#define SPI_BAUDRATEPRESCALER   (SPI_BaudRatePrescaler_4)

 //Set the priority of the SPI
 SPI_Init( SPI_FirstBit_MSB, SPI_BAUDRATEPRESCALER,
            SPI_Mode_Master, SPI_CPOL_Low, SPI_CPHA_1Edge,
            SPI_Direction_2Lines_FullDuplex, SPI_NSS_Soft);
 //Enable SPi
 SPI_Cmd(ENABLE); 
 
}/*spi_init*/


void nrf24l01_init(void)
{  
 spi_init();
 
 /*GPIOB::GPIO_Pin_4 -> SPI::NSS -> NRF24l01::CSN*/
 GPIO_Init( GPIOB, GPIO_Pin_4, GPIO_Mode_Out_PP_High_Fast);
 
 /*GPIOB::GPIO_Pin_3 -> NRF24l01::CE*/
 GPIO_Init( GPIOB, GPIO_Pin_3, GPIO_Mode_Out_PP_High_Fast);

 /*GPIOC::GPIO_Pin_0 -> NRF24l01::IRQ*/ 
 GPIO_Init(GPIOC, GPIO_Pin_0, GPIO_Mode_In_PU_IT); 
 EXTI_SetPinSensitivity(GPIO_Pin_0, EXTI_Trigger_Falling_Low);
 EXTI_ClearITPendingBit(GPIO_Pin_0);
}/*spi_init*/


The last stuff to do is to register an external interrupt function for nRF24L01's IRQ in file stm8_interrupt_vector.c.

 In my board, pin PC0 is the pin for nRF24L01 IRQ.  According to the STM8L101xx datasheet( in section "interrupt vector mapping", p.32), the PX0 interrupt number is 8.

Therefore, add the interrupt function in stm8_interrupt_vector.c :

struct interrupt_vector const _vectab[] = {
 {0x82, (interrupt_handler_t)_stext},
:
:
 {0x82, NonHandledInterrupt}, /* irq6  */
 {0x82, NonHandledInterrupt}, /* irq7  */
 
 {0x82, external_interrupt0_handler}, /* irq8  */
 
 {0x82, NonHandledInterrupt}, /* irq9  */


The interrupt function definition is:

@far @interrupt void external_interrupt0_handler(void)
{
 EXTI_ClearITPendingBit(GPIO_Pin_0); 
 esb_irq();
}/*external_interrupt0_handler*/

This very clear that once the interrupt is triggered, function esb_irq, which is as the same as nRF24LE1's, will be called to deal with the callback events of send/receive/max re-try.


It is all.



below is the uart output of this STM8L101F3PU + nRF24L01 while this device is communicating with nRF24LE (the original one, before this porting).


If I turn off the nRF24LE1, the uart output is :


It is compatible with the previous works totoally. Of course, this porting is able to communicate with my previous work on nRF5X.

2018年11月27日 星期二

nRF51/nRF52 Runs BLE Peripheral and ESB Protocol Concurrently


   The code for this post is here.   

  Continue my previous post about Nordic's Enhanced Shockburst™(ESB)  reliable transceiving, In this post I would like to discuss how to combine ESB with Bluetooth low energy(BLE).

       BLE protocol is widely adopted on the consumer mobile devices, ALL mainstream mobile operation systems support that, and ALL device have been implemented the function. If is extremely easy to find out a device supporting BLE at hand. The BLE network topology is point to point, it is very straightforward and intuitive. For the simplest topology, the connection setting is quick and instinct, easy to be accepted by dummy consumer. Thus, lots mobile peripherals use this protocol connecting with mobile to transfer data or fetch command.
     In most case, one to one structure is enough for the personal device, lots  sporting wrist/belt products have existed in the market.
     However, in home or small commercial cases, one to one structure is not enough, even enough, it is only adequate. You could try to consider a user scenario :  there is a large busking,  several installation separated in an performance stage, ,the performer needs to interactive those device. The controller needs to use phone or computer to ignite fire, spray gas.. etc. Besides, It is better for the stuffs that they could use phone to control those devices, because their working space is limited.

     WIFI is a solution for the small performance issue, but WIFI might be not a good solution, It is because the setting of WIFI is troublesome, and once the phone connect to a router which is not on internet, the phone is not able to access internet either (even the phone's LTE function running).

     ESB protocol provides star and mesh  topology, but the phones do not support ESB.  However, nRF51/nRF52chip is able to  run the two protocols at the same time, that fills the gap to use phone controlling or receiving data from multi-devices.

   In this demonstration, there is a nRF51/nRF52 device conveying BLE data to ESB and vice versa. For demonstrating the transfer in detail,  I also provide a device based on nRF24LE1 for sending/receiving ESB packets.

 
※ My code is based on nRF5X SDK 11, you could download that from here.

   零. Download the code from Github.
Place the whole folder under nRF5X_SDK11_ROOT\examples\ble_peripheral. The project files are as the same as SDK default examples, in pca10028 (nRF51) and pca10040 (nRF52) repectively. All projects are for Keil-C 5.
       nRF24LE1 device code is in under folder nRF24LE1, in this folder,  all required files have been included, you do not need to download nRFgo SDK.
        UART are in baudrate 38400 in all device.

 一. About the nRF5x code : 

   The main.c is based on ble_app_template, macro _INTACT denotes where I modified. I added:

    UART function for print output.
    A timer, to print time series per second.
    A BLE service, providing to send/receive data  to/from BLE central (phone).

 
   The code under esb and esb_timeslot folder I downloaded from here. notice that the code in esb folder is not the same as SDK default containing (nRF5X_SDK11_ROOT\components\properitary_rf\esb), the adopted IRQs have been changed. And the esb_timeslot\esb_timeslot.c and esb_timeslot\esb_timeslot.h have been modified for my application.

   Due to the IRQs collision might bring the ESB function hanging. Hance, as this thread mentioned, about line 467, esb_timeslot\esb_timeslot.c should be modified:


void nrf_esb_event_handler(nrf_esb_evt_t const * p_event)
{
    :
    :
    if (p_event->evt_id & NRF_ESB_EVENT_RX_RECEIVED)
    {
#if !defined(_INTACT)   
    fetch_esb_rx_data();
#else   
        /* Data reception is handled in a lower priority interrup. */
        /* Call UESB_RX_HANDLE_IRQHandler later. */
        NVIC_SetPendingIRQ(UESB_RX_HANDLE_IRQn);
#endif   
    }
}


The nrf_esb_event_handler function is called by function ESB_EVT_IRQHandler (esb/esb.c, line 909), which is a software IRQ. But UESB_RX_HANDLE_IRQn is the re-used of WDT_IRQ.  It is not appropriate to manipulate the other IRQ inside a non-software IRQ. Thus, it is necessary to modify the code for directly fetching ESB data  in the function nrf_esb_event_handler.


 二. The code nRF24LE1 :
  The code is copied from folder txrx of my previous post's repository. Note that function, about 100 of file esb_app.c :
 

hal_nrf_enable_dynamic_ack(false);

It should be consisted with  nRF5X code esb_timeslot/esb_timeslot.c, about line 523:


nrf_esb_config.selective_auto_ack = false;

If you set that as false in nRF24LE1 but leave that as true in nRF51 the UART output of nRF24LE1  would display rf_max_retry_count_reached  but the packet indeed delivered.  Disable dynamic_ack/auto_ack means you need to manually sendback ack packet(zero length packet) to sender.

  Result :

  In BLE scanner,  the peripheral name shows as "NRF51 ESB to/from ":
※ the device name in code has been changed as "ESB to/from BLE" for the original name was "NRF51 ESB to/from BLE", which is too long to display completely(over 20 bytes). But below images were captured before the fixing .





After the phone connected with the nRF5x and set notification(subscribing). the values would be received periodically and vary with time.
 



I sent a data as to the nRF5x:



And the nRF24LE1 would received this data, as its UART showing:




Reference :

Make Nordic Enhanced ShockBurst™ (ESB) Transceivers as Multiceiver and Reliable

Migration from µESB to nrf_esb

Running micro-ESB concurrently with BLE



2018年11月16日 星期五

Make Nordic Enhanced ShockBurst™ (ESB) Transceivers as Multiceiver and Reliable


    ※ This post focuses on nRF24LE1, if you are interested in how to use an independent microcontroller to control NRF24L01, please refer to my another post.

    ※ The multi-sending and multi-receiving code is here,  Receiver send back Acknowledge code is here.

      Nordic's Enhanced ShockBurst™ transceivers have been very widely adopted in IoT (Internet of things) industries or maker area, they feature affordable prices while being of mass deployment. Also the design references, documents, demonstration and example code are huge, tt is easy for users to use the transceivers and chips adapting their purposes.
      There are enormous example code of operating these transceivers, those code are on Arduno, STM32, MSP430...and so forth. In the most circumstances, the are ample for the users.
     But those code are few in mentioning "How to implement a RELIABLE multi-transmission". Even about the issue "how to operate a sender with designating a receiver",  the explanation is obscure. The lots discussion are written in words, the example code is lacked.
      The reliable transmission issue for are very essential for IoT purpose :  if the transmission is not reliable, it is difficult to plan a network topology.  If the mechanism of checking reliability does not be done in network-link layer, the mechanism must be implemented in application. It leads it is difficult to design such protocol, and the code be tedious.
      It is important also for the designated destination issue. If in network link layer, the sender is not able to specify which transceiver should receive.It  brings all device would receive the packets of the EACH sender, it results in, the firmware work be increased to filter out redundant data. Besides, if one transceiver receives too much packets simultaneously, it may be hanging.

    This post tries to demonstrate how to designated the receiver, and provide a example of the transmission with acknowledge.

  甲. Implement three firmware, one is able to multi-send with designating destination, another is able to multi-receive with designating source, the other is sender and receiver in one, but sending and receiving are from different address.

 乙. Implement a pair of firmware, the receiver is able to receive packet from six different addresses reliably.

 
   There are three famous Enhanced ShockBurst™ transceivers and chips in the market, I explain ahead:

壹. nRF24L01/nRF24L01+ : Pure transceiver, SPI communication with a microcontroller. (Note: nRF24L01 with plus does not support 250kb data rate)

貳. nRF24LE1  : An 8051 based microcontroller integrated Enhanced ShockBurst™ function, operate the communication via internal SPI.

參. nRF24LU1 : Similar with nRF24LE1, but this microcontroller has been integrated USB device function.

    My codes are based on nRF24LE1, but it is easy to port the code to the other microcontrollers.

一. implement a sender/receiver pair, which support multi-sending/receiving with designated destination and sources.


take look at  trscvr_para.h, those parameters must be the same for the transceivers which are  going to communicate each other. You should regard the ADDR number, those are adopted to set the pipe addresses:

#define ADDR_PIPE0_0        (0x10)
#define ADDR_PIPE0_1        (0x11)
#define ADDR_PIPE0_2        (0x12)
#define ADDR_PIPE0_3        (0x13)
#define ADDR_PIPE0_4        (0x14)


#define ADDR_PIPE1TO5_1        (0xE1)
#define ADDR_PIPE1TO5_2        (0xE2)
#define ADDR_PIPE1TO5_3        (0xE3)
#define ADDR_PIPE1TO5_4        (0xE4)

#define ADDR_PIPE1_0        (0x81)
#define ADDR_PIPE2_0        (0x82)
#define ADDR_PIPE3_0        (0x83)
#define ADDR_PIPE4_0        (0x84)
#define ADDR_PIPE5_0        (0x85)


The address structure for communication pipe, is as below (ref: nRF24L01+ Preliminary Product Specification v1.0, page 36):


PIPE0 ADDR_PIPE0_0 ADDR_PIPE0_1 ADDR_PIPE0_2 ADDR_PIPE0_3 ADDR_PIPE0_4
PIPE1 ADDR_PIPE1_0 ADDR_PIPE1TO5_1 ADDR_PIPE1TO5_2 ADDR_PIPE1TO5_3 ADDR_PIPE1TO5_4
PIPE2 ADDR_PIPE2_0 ADDR_PIPE1TO5_1 ADDR_PIPE1TO5_2 ADDR_PIPE1TO5_3 ADDR_PIPE1TO5_4
PIPE3 ADDR_PIPE3_0 ADDR_PIPE1TO5_1 ADDR_PIPE1TO5_2 ADDR_PIPE1TO5_3 ADDR_PIPE1TO5_4
PIPE4 ADDR_PIPE4_0 ADDR_PIPE1TO5_1 ADDR_PIPE1TO5_2 ADDR_PIPE1TO5_3 ADDR_PIPE1TO5_4
PIPE5 ADDR_PIPE5_0 ADDR_PIPE1TO5_1 ADDR_PIPE1TO5_2 ADDR_PIPE1TO5_3 ADDR_PIPE1TO5_4

Only pipe 0 have complete 5-byte address values ; the first to the fourth bytes of the others are the same, there are only zeroth bytes different.

The addressese should be the same for a pipe of a TX corresponding to a RX to make the communication work. (but the pipe numbers are not necessary to be the same)


The addresses are allocated to each pipe via below code(line 39 of esb_app_ptx_noack.c , line 36 of esb_app_txrx_noack.c and line and line ine 37 of esb_app_prx_noack.c ) :


{
 uint8_t i;
 for(i = HAL_NRF_PIPE0; i <= HAL_NRF_PIPE5; i++)
  hal_nrf_set_address(i, l_pipe_addr[i]);
}/*set all addresss*/


Specify a pipe as TX pipe (line 108 of esb_app_ptx_noack.c and line 128 of esb_app_txrx_noack.c) :
hal_nrf_set_address(HAL_NRF_TX, l_pipe_addr[tx_pipe_number]);


Specify the pipe(s) to receive packets(line 45 of esb_app_prx_noack.c and line 43 of esb_app_txrx_noack.c) :
hal_nrf_open_pipe(RX_PIPE, false); 

Open in here, does not mean "Enable the RF function", it is only to open the receiving function of that pipe; and the false in the argument, means to disable the auto-ack function.
If the pipe is only for sending data, it is not necessary to call the hal_nr_open_pipe function.


In my code, The PTX would send packet via pipe 5, and RX receive packet from pipe 5. UART, in baudrate 38400.

The UART outputs are :


PTX:
rf snd in pipe = 5, len = 6:: 05 da 00 20 eb bc
rf snd in pipe = 5, len = 6:: 05 db 00 20 ed b0
 2158 sec
rf snd in pipe = 5, len = 6:: 05 dc 00 20 ef a4
 2159 sec
rf snd in pipe = 5, len = 6:: 05 dd 00 20 f1 98
rf snd in pipe = 5, len = 6:: 05 de 00 20 f3 8c
 2160 sec
rf snd in pipe = 5, len = 6:: 05 df 00 20 f5 80

PRX:
rf rcv in pipe = 5, len = 6:: 05 be 00 20 b5 0c
rf rcv in pipe = 5, len = 6:: 05 bf 00 20 b7 00
 4645 sec
rf rcv in pipe = 5, len = 6:: 05 c0 00 20 b8 f4
rf rcv in pipe = 5, len = 6:: 05 c1 00 20 ba e8
 4646 sec
rf rcv in pipe = 5, len = 6:: 05 c2 00 20 bc dc

If you modify the RX_PIPE as HAL_NRF_PIPE0 or change the address value, and leave the TX codes intact, the output would be

 8 sec
 9 sec
 10 sec
 11 sec
 12 sec
 13 sec

That is, the PRX receiving nothing. Is is very natural, The pipe addresses of TX and RX  are not corresponding to each other.


If you add a device, which runs the TXRX code, and define the macro _RX_FOR_ALL_CHANNEL in PRX (in line 12 of  PRX\esb_app_prx_noack.h) and leave the PTX in working, the output would be :

TXRX :
rf snd in pipe = 3, len = 6:: 05 6c 00 00 d4 e4
rf rcv in pipe = 5, len = 6:: 05 cc 00 01 90 64
 55 sec
rf snd in pipe = 3, len = 6:: 05 6d 00 00 d6 d8
rf rcv in pipe = 5, len = 6:: 05 cd 00 01 92 58
rf snd in pipe = 3, len = 6:: 05 6e 00 00 d8 cc
rf rcv in pipe = 5, len = 6:: 05 ce 00 01 94 4c
 56 sec
rf snd in pipe = 3, len = 6:: 05 6f 00 00 da c0
rf rcv in pipe = 5, len = 6:: 05 cf 00 01 96 40

PRX:

rf rcv in pipe = 5, len = 6:: 05 4a 00 02 86 7c
rf rcv in pipe = 3, len = 6:: 05 ec 00 01 ce e4
rf rcv in pipe = 5, len = 6:: 05 4b 00 02 88 70
 76 sec
rf rcv in pipe = 3, len = 6:: 05 ed 00 01 d0 d8
rf rcv in pipe = 5, len = 6:: 05 4c 00 02 8a 64
rf rcv in pipe = 3, len = 6:: 05 ee 00 01 d2 cc
 77 sec
rf rcv in pipe = 3, len = 6:: 05 ef 00 01 d4 c0

The TXRX recive packet from pipe 5, and send the packets vis pipe 3. PRX receives the packets from the ALL pipes : the packets of PTX from pipe 5, the packets of TXRX from pipe 3.

     To here, you could try to think for a while for why Nordic calls communication pipe as pipe : it causes  a pipe, does not represent a device, it represents a communication channel of a corresponding sender/receiver.  If your topology needs, a pipe could be used for a group of sender/receiver (it is a bit like the "group" term in Zigbee).




二.  Make a sender/receiver pair, the communication in which is reliable : there sender would know if the receiver is indeed received the packet or not.


The key to enable auto-acknowledge mechanism, is to set the corresponding address as HAL_NRF_TX and HAL_NRF_PIPE0, and open the two pipe with auto-ack function(about line 46 of PTX/esb_app_ptx_ack.c).


 
 {
  uint8_t i;
  for(i = HAL_NRF_PIPE0; i <= HAL_NRF_PIPE5; i++)
   hal_nrf_set_address(i, l_pipe_addr[i]);
 }/*set all addresss*/

#ifndef _TX_FOR_ALL_CHANNEL  
 hal_nrf_set_address(HAL_NRF_PIPE0, l_pipe_addr[TX_PIPE]); 
 hal_nrf_set_address(HAL_NRF_TX, l_pipe_addr[TX_PIPE]);
 
 hal_nrf_open_pipe(HAL_NRF_PIPE0, true);
 hal_nrf_open_pipe(TX_PIPE, true);
#endif

it is necessary to open the pipe 0 is for receiving ack signal .


If you would like to append some data to the acknowledge packet from receiver side, you must call the 2 function (about line 65 of PTX/esb_app_ptx_ack.c) :

 hal_nrf_enable_dynamic_ack(true);
 hal_nrf_enable_ack_payload(true);

In sender's RF interrupt request (RF-IRQ) function, you could determine if it is ack-appending or not though which pipe receives. The determination I adopted in my code  about line 131 of PTX/esb_app_ptx_ack.c. Pay attention: the ack packets are from pipe 0 always :


if(irq_flags & (1<<HAL_NRF_RX_DR))
{ 
 //ack payload received
 uint8_t received_pipe; 
 if(false == hal_nrf_rx_fifo_empty())
 {
  uint16_t pipe_and_len;
  pipe_and_len = hal_nrf_read_rx_payload(&l_rx_payload[0]);
  
  received_pipe = (uint8_t)(pipe_and_len >> 8);
  l_rx_payload_len = (uint8_t)(0xff & pipe_and_len);
 }/*if */
 
 hal_nrf_flush_rx();
 if(HAL_NRF_PIPE0 == received_pipe)
  l_is_ack_reached_flag = 1;
  
}/*if */


The judgment is very important while you want to implement a device of sender/receiver as one (as the TXRX folder) .


In the receiver side, the ack-packet will be sendbacked via the function:

void hal_nrf_write_ack_payload(uint8_t pipe, const uint8_t *tx_pload, uint8_t length);

the pipe in argument must be as the same as the receiving pipe. You could refer to function esb_send_ack_data, in PRX/esb_app_prx_ack.c, line 147.


After preliminary explanation, I show their UART output for more detail clarification:

PTX :


rf snd in pipe = 3, len = 6:: 05 38 04 12 15 00
ack received ::  6a 6a 6a 6a
 2205 sec
rf snd in pipe = 3, len = 6:: 05 39 04 12 15 00
ack received ::  6b 6b 6b 6b
rf snd in pipe = 3, len = 6:: 05 3a 04 12 15 00
ack received ::  6c 6c 6c 6c
 2206 sec
rf snd in pipe = 3, len = 6:: 05 3b 04 12 15 00
ack received ::  6d 6d 6d 6d

PRX:

rf rcv in pipe = 3, len = 6:: 05 37 04 12 15 7d
ack len = 4::  6a 6a 6a 6a
 3000 sec
rf rcv in pipe = 3, len = 6:: 05 38 04 12 15 7d
ack len = 4::  6b 6b 6b 6b
rf rcv in pipe = 3, len = 6:: 05 39 04 12 15 7d
ack len = 4::  6c 6c 6c 6c
 3001 sec
rf rcv in pipe = 3, len = 6:: 05 3a 04 12 15 7d
ack len = 4::  6d 6d 6d 6d

The PTX and PRX are consisted with each other. PTX sends a serial number  to PRX, PRX acknowledges with another series of number.

If you turn off the PRX device, PTX would show the warning of retry_count_reached. It is, HAL_NRF_MAX_RT IRQ  has been triggered (about line 105 of PTX/esb_app_ptx_ack.c) :  The retry count has been reached but the packet does not been sent successfully yet.


 2986 sec
rf snd in pipe = 3, len = 6:: 05 53 04 12 15 00
rf_max_retry_count_reached!!
rf snd in pipe = 3, len = 6:: 05 54 04 12 15 00
rf_max_retry_count_reached!!
 2987 sec
rf snd in pipe = 3, len = 6:: 05 55 04 12 15 00
rf_max_retry_count_reached!!


But if HAL_NRF_PIPE0  do not be open with auto_ack (hal_nrf_open_pipe(HAL_NRF_PIPE0, true) ), this HAL_NRF_MAX_RT IRQ would never been triggered even the max-retry count has been reached.

Max retry count and trying-interval are able to be set via the function:


void hal_nrf_set_auto_retr(uint8_t retr, uint16_t delay);

I called it at line 7 of PTX/esb_app_tx_ack.c.


     In this example, I assume all sending event request reciever's ack signal. If your sender suddenly does not need the ack from the receiver, you could call the function for sending the packet:

void hal_nrf_write_tx_payload_noack(const uint8_t *tx_pload, uint8_t length);

Calling this function would set the no_ack flag of ESB packet as true. Once the  no_ack flag has been set, The receiver would not sendback the auto-ack signal to sender.
     (The dynamical ack function, hal_nrf_enable_dynamic_ack, must be configured before calling this function, as line65, PTX/esb_app_ptx_ack.c,  More detail about the ESB packet format, please refer to here)

     However, in the application layer, there is NO WAY for the receiver to query if the no_ack flag in received packet has been set or not. If your receiver sendbacks an ack_payload to a sender requesting no_ack,  your sender would not received that.
     A solution to this issue is to adopt different pipe specified for ack/no_ack transmission, which is Nordic's suggestion. The other solution I know, is to give a bit in payload to describe if the packet needs ack-payload or not.


NOTE :

For PTX, you could change macro TX_PIPE to the other pipe, or enable macro _TX_FOR_ALL_CHANNEL for sending each channel by turns. There is also macro PX_PIPE/_PX_FOR_ALL_CHANNEL for PRX, those functions are similar to PTX's.

Nordic's doc said the Constant ESB_MAX_ACK_PAYLOAD_LEN, is 32 bytes, but based on my test, the length is 4 bytes only. If you know how to effectively sendback a packet of 32 bytes appended with acknowledge, please tell me. But from my experience, 4 bytes is very enough for appending to acknowledge in a good IoT architecture.


2018年8月3日 星期五

Exploit Scapy to Generate WIFI Probe Request Packets on x86 Linux Distribution and OpenWRT


      Thought there are discussing threads and posts about how to use scapy library to generate probe request packets, but based on my searching, all of those are not organized and detail. This post would like to fill the gap.

   I assume you have install the python and scapy in your linux, if you have not done that, please refer to those:

Python: http://ubuntuhandbook.org/index.php/2017/07/install-python-3-6-1-in-ubuntu-16-04-lts/
Anaconda :  https://conda.io/docs/user-guide/install/linux.html  (optional)
Scapy : https://scapy.readthedocs.io/en/latest/installation.html#installing-scapy-v2-x

一. Broadcasting the probe-request in the x86-based Linux distribution:

below is the python code, named as broadcastproberequest.py,  to call scapy for generating probe-request:

#!/usr/bin/python
#coding:utf-8

import time
from scapy.all import *


def KeepBroadcastingProbeRequest(interface="wlan0", interval=0.1, \
        duration=30, ssid="譖路由器"):
    
    count = duration/interval

    packet = RadioTap()/Dot11(type=0, subtype=4, \
    addr1="ff:ff:ff:ff:ff:ff", addr2="00:11:22:33:44:55" ,addr3="ff:ff:ff:ff:ff:ff") \
    /Dot11Elt(ID="SSID", info=ssid)

    sendp(packet,  iface= iface, inter=interval, count=count)


if __name__ == '__main__':

    iface = 'wlan0'
    interval = 0.1
    duration = 30
    ssid = "開源路由器"
    try:
        opts, args = getopt.getopt(sys.argv[1:],"i:t:s:d:")
    except getopt.GetoptError as e:
        print("-i <iface>  -t <interval> -s <SSID> -d <duration>")
        sys.exit(-1)

    for opt, arg in opts:
        if opt == '-i':
            iface = arg
        elif opt == '-t':
            interval = float(arg)
        elif opt == '-s':
            ssid = arg   
        elif opt == '-d':
            duration = int(arg)
        elif opt == '-h':
            print("-i <iface>  -t <interval> -s <SSID> -d <duration>")
            sys.exit(1)

    print( "iface=%s, interval=%1.3f, duration=%d, SSID=%s" \
        %(iface, interval, duration, ssid))

    KeepBroadcastingProbeRequest(interface=iface, interval=interval, ssid=ssid, duration=duration)

Before you run your code,  if you currently use this wifi-interface (network card) online,  YOU SHOULD DISCONNECT IT AND SET THIS INTERFACE AS MINOTORING MODE :

Inquiry the network-interfaces statue:
gaiger@i5-3210M:~$ ifconfig
:
wlp3s0    Link encap:Ethernet  HWaddr 60:67:20:a5:44:ae  
          inet addr:192.168.1.140  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fd80:d8b6:da22:0:69c3:1d2c:cf5:ce18/64 Scope:Global
          inet6 addr: fd80:d8b6:da22:0:818b:1644:e8b:d756/64 Scope:Global
          inet6 addr: fd80:d8b6:da22:0:a068:21d:dfe8:f717/64 Scope:Global
          inet6 addr: fd80:d8b6:da22:0:2cae:5b7f:d19e:1280/64 Scope:Global
          inet6 addr: fd80:d8b6:da22::a34/128 Scope:Global
          inet6 addr: fd80:d8b6:da22:0:3b50:b57d:b0c8:4a68/64 Scope:Global
          inet6 addr: fd80:d8b6:da22:0:99e4:c8ee:ea9a:bfe8/64 Scope:Global
          inet6 addr: fe80::f19c:8dd9:2058:c699/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1337204 errors:0 dropped:555386 overruns:0 frame:0
          TX packets:543649 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:528846681 (504.3 MiB)  TX bytes:79969190 (76.2 MiB)


Inquiry the wireless interface statue:
gaiger@i5-3210M:~$ iwconfig
wlp3s0    IEEE 802.11abgn  ESSID:"Tenda 3270 "  
          Mode:Managed  Frequency:2.437 GHz  Access Point: C8:3A:35:78:2D:98   
          Bit Rate=72.2 Mb/s   Tx-Power=15 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on
          Link Quality=70/70  Signal level=-35 dBm  
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:13   Missed beacon:0
:


The WIFI-interface in this computer is named as wlp3s0. I DISCONNECT IT FROM ROUTER and set it as monitoring mode in channel 1( 2.412 GHz):
gaiger@i5-3210M:~$ sudo ifconfig wlp3s0 down
gaiger@i5-3210M:~$ sudo iwconfig wlp3s0 mode monitor
gaiger@i5-3210M:~$ sudo iwconfig wlp3s0 channel 1
gaiger@i5-3210M:~$ sudo ifconfig wlp3s0 up
gaiger@i5-3210M:~$ sudo iwconfig wlp3s0
wlp3s0    IEEE 802.11abgn  Mode:Monitor  Frequency:2.412 GHz  Tx-Power=15 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on


Of course, I could set the other channels, below is the result I set it as channel 6.
wlp3s0    IEEE 802.11abgn  Mode:Monitor  Frequency:2.437 GHz  Tx-Power=15 dBm   
          Retry short limit:7   RTS thr:off   Fragment thr:off
          Power Management:on

Now the code is able to be workable:
gaiger@i5-3210M:~/sandbox/python-spider$ sudo python broadcastproberequest.py -i mon0 -t 0.02 -s 偽測探 -d 300


  I use the other device(ESP8266), to achieve the sniffing purpose, it is the sniffed result:


About the arguments :

-i <iface> -t <interval> -s <SSID> -d <duration>
 iface : interface name, default is wlan0
interval :  broadcasting interval, in unit of second, default is 0.1.
SSID : the fake ssid you would like to broadcast, it supports UTF-8 for international purpose. Default is 開源路由器
duration : how long would the broadcasting last, in unit of second. default is 30 seconds.


二. Run the code in OpenWRT.

  甲. Check there is over 7.5 MB storage space available on your OpenWRT device :

root@JoySince:~# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                   12.3M      2.7M      9.6M  22% /
:

If your space is not adequate, you could try to adopt extroof skill.

 乙. Install those package, libffi, python-mini and python:
root@JoySince:~#opkg update
root@JoySince:~#opkg install libffi
root@JoySince:~#opkg install python-mini
root@JoySince:~#opkg install python


丙. install scapy.

   Before install scapy, you need to install tcpdump and unzip.
root@JoySince:~#opkg install python tcpdump unzip

After the installation has done, download the scapy from here and move the downloaded scapy.XXX.tar.gz file to your OpenWRT.

install the scapy.XXX.tar.gz file (mine is scapy-2.4.0.tar.gz), I assume the scapy.XXX.tar.gz is located in /tmp folder:

root@JoySince:/tmp~#tar -zxvf scapy-2.4.0.tar.gz
root@JoySince:/tmp~#cd scapy-2.4.0
root@JoySince:/tmp~#python setup.py install

After the installation has been done, you could remove the tar.gz file and the unzip scapy folder.


丁.  Set the WIFI interface on OpenWRT in monitor mode.

Modify the file /etc/config/wireless as :
config wifi-device 'radio0'
        option type 'mac80211'
#       option channel '8'
        option channel '1' ##
        option hwmode '11g'
        option path 'platform/ar933x_wmac'
        option htmode 'HT40'
        option txpower '30'
        option country 'US'

config wifi-iface
        option device 'radio0'
        option mode monitor ##
#       option mode 'ap'
#       option ssid 'JoySince'
        option network 'lan'
        option encryption 'psk'
        option key '12345678'

The lines begins from # are what I commented out, the lines end in ## are what I added.
After the modification has been done, set this configuration applied:
root@JoySince:/tmp# wifi
root@JoySince:/tmp# iw wlan0 info
Interface wlan0
        ifindex 6
        wdev 0x2
        addr 00:ca:01:06:0d:da
        type monitor
        wiphy 0
        channel 1 (2412 MHz), width: 20 MHz (no HT), center1: 2412 MHz

If it does not work, reboot your OpenWRT.


丁. Move the code broadcastproberequest.py to your OpenWRT,  and run the code.

root@JoySince:~# python broadcastproberequest.py -d 300  -t 0.05 -s 白羅剎


Result :


※ You could adopt the other OpenWRT with the same configuration of file /etc/config/wireless to monitor the broadcasted probe-request packets :

root@JoySince:~# tcpdump -i wlan0 -e -s 1024 type mgt subtype probe-req -vv

But you could not use ONLY ONE openWRT to broadcast and monitor probe-request  at the same time.

2018年4月5日 星期四

ESP8266 RTOS : Device Firmware Update Over the Air (DFU-OTA) via the Private Server

      ESP8266 is a very popular micro-controller featuring WIFI with its bargaining price. There are some product based on ESP8266/ESP8285 (ESP8285 = ESP8266 embedded 1 MB flash) for home IoT using. The makers enjoy remolding the firmware of the products to fix their personal purposes.
      In most case, the maker re-program ESP8266 in the framework of Arduino, thus, there are lots post and discussion about how to deal with DFU-OTA based on ESP8266 Arduino. However, The information about the same goal but the ESP8266 based on RTOS framework is limited, undocument or incomplete. This post wants to fill the gap.
   By default, DFU-OTA server is based on Espressif: It forces the ESP8266 to be on internet. But some use case the ESP8266 would not be connected with internet (maybe for the security issue). Therefore, private DFU-OTA is very necessary. In here I assume your DFU-OTA server is linux based, But it is easy to port the DFU-OTA function to the other platform, for the DFU-OTA working principle is typical TCP-IP.

Note: this example assumes your esp8266 is within 1MB flash!  It may work in otherwise size or not I am not sure.


零. Download the requisite library:
     In here, You should download the include/upgrade.h,  user/upgrade_lib.c and user/fota_crc32.c., and move them into your ESP8266 project folder. I assume the files place under the sub-folder, dfu-ota, of your project folder.
   
一. Create a c file,  named  dfu_ota.c, under the same sub-folder, the code be :

#include "esp_common.h"
#include "espconn.h"


#include "dfu_ota.h"
#include "upgrade.h"



#define LOCAL     static

LOCAL struct espconn g_tcp_client;
LOCAL esp_tcp g_tcp;
LOCAL int g_firmware_length = 0;

enum 
{
 TCP_CONNECTING   = 0x00,
 SENT_TARGET_SECTION  = 0x01,
 RECEIVED_LENGTH   = 0x02,
 KEEP_RECEIVING_DATA  = 0x03,
 RECEIVED_DATA   = 0x04,

 TCP_DISCONNECTED = 0xff,
};


LOCAL int g_sending_state = TCP_DISCONNECTED;


int firmware_upgrading_fail(void)
{ 
 printf("upgrading fail\r\n");
 espconn_disconnect(&g_tcp_client);

 printf("reboot the system\r\n");
 system_restart();
}/*firmware_upgrading_fail*/


//LOCAL uint8_t g_tcp_server_ip[4] = {0};
LOCAL uint8_t g_tcp_retry_connection_count = 0;

LOCAL void tcp_connection_error_callback(void *arg, int8_t err)
{
 struct espconn *p_tcp_server;
 p_tcp_server = arg;

 printf("tcp connection err = %d\r\n", err);

 if(ESPCONN_OK != err)
 {
#define MAX_RETRY_CONNECTION_COUNT   (5)

  if(MAX_RETRY_CONNECTION_COUNT > g_tcp_retry_connection_count)
  {
   printf("retry the tcp connection...\r\n");
   delay_ms(1000);
   g_tcp_retry_connection_count++;
   espconn_connect(&g_tcp_client);
  }
  else
  {
   printf("relinquish this upgrading\r\n");
   g_tcp_retry_connection_count = 0;
   firmware_upgrading_fail();
  }/*if */
 }/*if ESPCONN_OK */

}/*tcp_connection_error_callback*/


LOCAL void tcp_connect_callback(void *arg)
{
 struct espconn *p_tcp_server;
 uint8_t tcp_state;
 uint8_t target_section;
 p_tcp_server = arg;

 if(USER_BIN1 == system_upgrade_userbin_check())
 {
  target_section = 2;
  printf("current firmware in user1\r\n");
 }
 else
 {
  target_section = 1;
  printf("current firmware in user2\r\n");
 }

 printf("fetch firmware%d\r\n", target_section);

 tcp_state = espconn_send(p_tcp_server, &target_section, sizeof(uint8_t));
 printf("target_section sending\r\n");
 g_sending_state = SENT_TARGET_SECTION;
#define INTERVAL_BETWEEN_PACKETS_IN_MSEC  (10)

 delay_ms(INTERVAL_BETWEEN_PACKETS_IN_MSEC);
}/*tcp_connect_callback*/


LOCAL void tcp_sent_to_buffer_callback(void *arg)
{
 struct espconn *p_tcp_server;
 uint8_t tcp_state;

 p_tcp_server = arg;
}/*tcp_sent_to_buffer_callback*/


#define OTA_PACKET_SIZE     (1280)
LOCAL int g_remained_firmware_length = 0;

LOCAL void tcp_received_callback(void *arg, char *p_data, unsigned short len)
{
 struct espconn *p_tcp_server;
 uint8_t tcp_state;
 uint8_t data_buffer[OTA_PACKET_SIZE];

 p_tcp_server = arg;

 switch(g_sending_state)
 {
  case SENT_TARGET_SECTION:
  if(sizeof(int) != len)
  {
   printf("received length error\r\n");
   goto Flag_Error_Occurring;
  }/*if */

  g_sending_state = RECEIVED_LENGTH;
  memcpy(&g_firmware_length, p_data, sizeof(int));
  printf("firmware size = %d\r\n", g_firmware_length);

  if(g_firmware_length < 0)
  {
   printf("firmware_length error\r\n");
   goto Flag_Error_Occurring;
  }
  g_remained_firmware_length = g_firmware_length;
  break;


  case RECEIVED_LENGTH:
  case KEEP_RECEIVING_DATA:

  memcpy(&data_buffer[0], p_data, len);
  g_remained_firmware_length -= len;

  if(0 > g_remained_firmware_length)
  {
   printf("received data size is not marched\r\n");
   goto Flag_Error_Occurring;
  }

  printf("received firmware data %d bytes, remain %d bytes...\r\n", 
   g_firmware_length - g_remained_firmware_length, g_remained_firmware_length);

  if(false == system_upgrade(&data_buffer[0], len))
  {
   printf("\r\nsystem_upgrade error\r\n");
   goto Flag_Error_Occurring;
  }/*if false == system_upgrade*/


  if(0 == g_remained_firmware_length)
  {
   printf("\r\n firmware has been downloaded done\r\n");
   g_sending_state = RECEIVED_DATA;

   if( false == upgrade_crc_check(system_get_fw_start_sec(), g_firmware_length))
   {
    printf("firmware upgrading FAIL : CRC is not consisted!!\r\n");
    system_upgrade_flag_set(UPGRADE_FLAG_IDLE);
    espconn_disconnect(&g_tcp_client);
   }
   else
   {
    printf("firmware upgrading is successful.\r\n");
    system_upgrade_flag_set(UPGRADE_FLAG_FINISH);
    espconn_disconnect(&g_tcp_client);
   }/* check crc*/
  }
  else
  {
   system_upgrade_flag_set(UPGRADE_FLAG_IDLE);
   g_sending_state = KEEP_RECEIVING_DATA;
  }
  break;
 default:
  break;
 }
 return;


Flag_Error_Occurring:
 firmware_upgrading_fail();
 return;
}/*tcp_received_callback*/


LOCAL void tcp_disconnect_callback(void* arg)
{
 printf("ota server disconnected\r\n");
 g_sending_state = TCP_DISCONNECTED;

 system_upgrade_deinit();
 if (UPGRADE_FLAG_FINISH == system_upgrade_flag_check())
 {
  printf("reboot the system for new firmware\r\n");
  system_upgrade_reboot();
 }
 else
 {
  printf("upgrade firmware fail\r\n");
  system_restart();
 }

}/*tcp_disconnect_callback*/



int device_firmware_upgrade_over_the_air(uint8_t *p_server_ip)
{
 uint8_t con_status;
 uint8_t tcp_state;

 printf("start device firmware upgrading over the air...\r\n");


 con_status = wifi_station_get_connect_status();
 
 if (con_status != STATION_GOT_IP) 
 {
  printf("error, this endpoint has not be connected with router.\r\n");
  printf("relinquish this firmware upgrading.\r\n");
  firmware_upgrading_fail();
  return -1;
 }/*if*/


#define TCP_SERVER_REMOTE_PORT       (5555)
 g_tcp.remote_port = (TCP_SERVER_REMOTE_PORT);
 g_tcp_client.type = ESPCONN_TCP;
 g_tcp_client.proto.tcp = &g_tcp;

 memcpy(g_tcp.remote_ip, p_server_ip, 4*sizeof(uint8_t));

 espconn_regist_reconcb(&g_tcp_client, tcp_connection_error_callback);
 espconn_regist_connectcb(&g_tcp_client, tcp_connect_callback);
 espconn_regist_sentcb(&g_tcp_client, tcp_sent_to_buffer_callback);
  espconn_regist_recvcb(&g_tcp_client, tcp_received_callback);
 espconn_regist_disconcb(&g_tcp_client, tcp_disconnect_callback);

 system_upgrade_flag_set(UPGRADE_FLAG_START);
 system_upgrade_init();


#define RELATIVE_END_SECTOR_NUM     (0x80 - 0x0c)
 {
  uint16_t begin_sector;
  uint16_t i;
  printf("erasing flash sector to store new firmware, it takes a while...\r\n");
  begin_sector = system_get_fw_start_sec();
  printf("erasing flash sector from 0x%02x to 0x%02x\r\n", begin_sector, 
   begin_sector + RELATIVE_END_SECTOR_NUM - 1);

  for(i = 0; i< RELATIVE_END_SECTOR_NUM; i++)
   spi_flash_erase_sector(begin_sector + i);
 }/*erase flash*/


 tcp_state = espconn_connect(&g_tcp_client);
 g_sending_state = TCP_CONNECTING;

 if(0 != tcp_state)
 {
  g_sending_state = TCP_DISCONNECTED;
  printf("espconn_connect error, give up this connection\r\n");
  firmware_upgrading_fail();
 }/*if 0*/

 return 0;
}/*device_firmware_upgrade_over_the_air*/


The corresponding header dfu_ota.h placed in the same folder, is,

#ifndef _DFU_OTA_H_
#define _DFU_OTA_H_


int device_firmware_upgrade_over_the_air(uint8_t *p_server_ip);

#endif

To here, the content under dfu_ota folder  be :

$ ls dfu_ota/
dfu_ota.c  dfu_ota.h  fota_crc32.c  Makefile  upgrade.h  upgrade_lib.c

The Makefile under dfu_ota folder, copied from common sub-folder, is with the modification, about line 14:


ifndef PDIR
GEN_LIBS = libdfu_ota.a
endif

The Makefile under your project's root folder, Should be added the dfu_ota content:

SUBDIRS=    \
        user    \
        :
        dfu_ota
:
:
COMPONENTS_eagle.app.v6 = \
        user/libuser.a  \
        :
        dfu_ota/libdfu_ota.a


二. DFU-OUA server code.

The DFU-OTA server would be responding the DFU requesting, the code, dfu_ota_server.c be:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <string.h>

#include <linux/limits.h>

//#include <inttypes.h>

 #include <errno.h>

#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>

#include <signal.h> 

#include <sys/stat.h>

/*for show ip*/
#include <arpa/inet.h>
#include <sys/socket.h>
#include <netdb.h>
#include <ifaddrs.h>

#include <pthread.h>

#ifdef __mips__
 #include <byteswap.h>
#endif

#include <sys/time.h>


#define PORT_NUMBER       (5555)

#define ONE_PACKET_MAX_SIZE     (4*1024)

#define LOCAL         static

#define INVALID_SOCKET      (int)(~0)
#define MAX_CLIENT_NUMER     (32)



void delay_micro_sec(unsigned int delay_time_in_us)
{
 struct timeval now;
 struct timeval period;
 struct timeval end;

 gettimeofday(&now, NULL);

 period.tv_sec = delay_time_in_us / 1000000;
 period.tv_usec = delay_time_in_us % 1000000;

 timeradd(&now, &period, &end);

 while(timercmp(&now, &end, < ))
  gettimeofday(&now, NULL);

}/*delay_micro_sec*/


LOCAL int g_master_socket;
LOCAL int g_client_socket;

int esp8266_ota_server_start(char **pp_firmware_location)
{
 int i;
 struct sockaddr_in server_addr;

 if(NULL == pp_firmware_location[0] || NULL == pp_firmware_location[1])
 {
  return -1;
 }

 g_master_socket = socket(AF_INET , SOCK_STREAM , 0);

 if(-1 == g_master_socket)
 {
  printf("Could not create socket\r\n");
  return -1;
 }/*if*/

 //Prepare the sockaddr_in structure
 server_addr.sin_family = AF_INET;
 server_addr.sin_addr.s_addr = INADDR_ANY;
 server_addr.sin_port = htons(PORT_NUMBER);

 {
  int reuseaddr;
  int reuseaddr_len;
  reuseaddr = 1;
  reuseaddr_len = sizeof(reuseaddr);
  setsockopt(g_master_socket, SOL_SOCKET, SO_REUSEADDR, &reuseaddr, reuseaddr_len);
 }/*local variabe */

 //Bind
 if(0 > bind(g_master_socket,(struct sockaddr *)&server_addr , sizeof(server_addr)) )
 {
  perror("bind failed. Error\r\n");
  return -2;
 }/*if bind*/
 printf("ota server is ready to be connected\r\n\r\n");


#define MAX_PENDING_CONNECTION_NUMBER   (3)

 listen(g_master_socket , MAX_PENDING_CONNECTION_NUMBER);

 while(1)
 {
  struct sockaddr_in client_addr;
  int socket_len;

  int written_len;
  unsigned int firmware_size;
  int is_succ_sent;

  unsigned char target_firmware_section;

  g_client_socket = accept(g_master_socket, 
   (struct sockaddr *)&client_addr, (socklen_t*)&socket_len);

  if(INVALID_SOCKET == g_client_socket)
  {
   printf("accepted ERROR\r\n");
   continue;
  }/*if */

  printf("\r\n\r\n%s connected\r\n", inet_ntoa(client_addr.sin_addr));

  is_succ_sent = 0;
  {
   socklen_t addr_len;

   addr_len = sizeof(client_addr);
   getpeername(g_client_socket, (struct sockaddr*)&client_addr, &addr_len);
  }/*local variable*/


  {
   struct timeval timeout_val;

#define MAX_ONE_PACKET_TIMEOUT_IN_SEC    (10)
   timeout_val.tv_sec = MAX_ONE_PACKET_TIMEOUT_IN_SEC;
   timeout_val.tv_usec = 0;

   setsockopt(g_client_socket, SOL_SOCKET, SO_SNDTIMEO, 
    &timeout_val, sizeof(timeout_val));

   setsockopt(g_client_socket, SOL_SOCKET, SO_RCVTIMEO, 
    &timeout_val, sizeof(timeout_val));
  }/*local variables*/

  {

   unsigned char read_buffer[16];
   int read_size;

   read_size = read(g_client_socket, &read_buffer[0], sizeof(read_buffer));
   if(1 != read_size)
   {
    printf("received firmware region index size error\r\n");
    goto Flag_end_connection;
   }
   target_firmware_section = 0;
   memcpy(&target_firmware_section, &read_buffer[0], sizeof(unsigned char));


   if(!(1 == target_firmware_section || 2 == target_firmware_section) )
   {
    printf("received firmware region index is not valid\r\n");
    printf("firmware_region_index = %d\r\n", (int)target_firmware_section);
    goto Flag_end_connection;
   }/*if*/

   printf("firmware%d would be transmitted to endpoint\r\n", (int)target_firmware_section);
  }/*local variable*/

  {
   struct stat fw_dir_st = {0};

   if(-1 == stat(pp_firmware_location[target_firmware_section - 1], &fw_dir_st))
   {
    printf("ERROR :: %s does not exist!\r\n", 
     pp_firmware_location[target_firmware_section - 1]);

    goto Flag_end_connection;
   }/*if */

   firmware_size = (int)fw_dir_st.st_size;
   printf("firmware%d size = %3.1f KB\r\n", target_firmware_section, firmware_size/1024.0);
  }/*local variable*/

  {
   unsigned int firmware_size_le;
#if __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
   firmware_size_le = __bswap_32(firmware_size);
#else
   firmware_size_le = firmware_size;
#endif
   written_len = write(g_client_socket, &firmware_size_le, sizeof(unsigned int));

   if(sizeof(unsigned int) != written_len)
   {
    printf("\r\nsend firmware length error\r\n");
    goto Flag_end_connection;
   }/*if */
  }/*send data length*/

 {
  unsigned int remain_firmware_size;
  FILE *fp;

  remain_firmware_size = firmware_size;

  fp = fopen(pp_firmware_location[target_firmware_section - 1], "rb");

  if(NULL == fp)
  {
   printf("open file %s error\r\n", 
    pp_firmware_location[target_firmware_section - 1]);
   goto Flag_end_connection;
  }/*if */

  while(remain_firmware_size > 0)
  {
#define OTA_PACKET_SIZE       (1280)
   unsigned char data_buffer[OTA_PACKET_SIZE];
   unsigned int sending_size;

   sending_size = fread(&data_buffer[0], 1, sizeof(data_buffer), fp);
   written_len = write(g_client_socket, &data_buffer[0], sending_size);

   if(sending_size != written_len)
   {
    printf("\r\nsend firmware data error\r\n");
    goto Flag_end_connection;
   }/*if*/

   remain_firmware_size -= written_len;

   printf("\rfirmware data has sent %3.1f KB, remain %3.1f KB   ", 
    (firmware_size - remain_firmware_size)/1024.0, remain_firmware_size/1024.0);
   fflush(stdout);

#define PACKET_INTERVAL_IN_MSEC     (400)
   delay_micro_sec(PACKET_INTERVAL_IN_MSEC*1000);
  }/*while*/

  fclose(fp); fp = NULL;
  printf("\r\n");

  is_succ_sent = 1;
 }/*send data */

 printf("firmware %3.1f KB has been sent successfully    \r\n", firmware_size/1024.0);

Flag_end_connection:
 if(0 == is_succ_sent)
  printf("WARNING:: firmware upgrading has been FAIL\r\n");

 close(g_client_socket); g_client_socket = INVALID_SOCKET;
 }/*while*/

 close(g_master_socket);
 return 0;
}/*esp8266_ota_server_start*/


int esp8266_ota_server_end(void)
{
 if( INVALID_SOCKET != g_client_socket)
  close(g_client_socket);
 g_client_socket = INVALID_SOCKET;

 if(INVALID_SOCKET != g_master_socket)
  close(g_master_socket);
 g_master_socket = INVALID_SOCKET;

 return 0;
}/*esp8266_ota_server_end*/


void InterruptHandler(int sig)
{
 printf("\r\n__FUNCTION__ = %s\r\n", __FUNCTION__);
 esp8266_ota_server_end();

 exit(0);
}/*InterruptHandler*/


int main(int argc, char *argv[])
{
 extern char *__progname;
 int k;
 char firmware_location[2][PATH_MAX];
 char *p_firmware_location[2];

 signal(SIGINT, InterruptHandler); 

 printf("\r\n%s\r\n", __progname);

 for(k = 0; k < 2; k++){
  memset(&firmware_location[k][0], 0, PATH_MAX);
  p_firmware_location[k] = NULL;
 }/*for k*/

 k = 1;
 while(k < argc)
 {
  if(0 == strncmp("-fw1", argv[k], strlen("-fw1")) 
   || 0 == strncmp("--user1", argv[k], strlen("-user1")) )
  {
   if(k + 1 < argc)
   {
    strncpy(&firmware_location[0][0], argv[k + 1], PATH_MAX);
    p_firmware_location[0] = &firmware_location[0][0];
   }
   else
   {
    printf("ERROR :: -fw1/--user1 should be followed by firmware location.\r\n");
    return -1;
   }/*if */
  }/*if */

  if(0 == strncmp("-fw2", argv[k], strlen("-fw1")) 
   || 0 == strncmp("--user2", argv[k], strlen("-user1")) )
  {
   if(k + 1 < argc)
   {
    strncpy(&firmware_location[1][0], argv[k + 1], PATH_MAX);
    p_firmware_location[1] = &firmware_location[1][0];
   }
   else
   {
    printf("ERROR :: -fw1/--user2 should be followed by firmware location.\r\n");
    return -1;
   }/*if */
  }/*if */

  k++;
 }/*while*/


 if(NULL == p_firmware_location[0])
 {
  printf("-fw1 should be specified\r\n");
  return -1;
 }/*if non specified fw1 location*/

 if(NULL == p_firmware_location[1])
 {
  printf("-fw2 should be specified\r\n");
  return -1;
 }/*if non specified fw2 location*/


 for(k = 0; k < 2; k++)
 {
  struct stat fw_dir_st = {0};

  if(-1 == stat(p_firmware_location[k], &fw_dir_st))
  {
   printf("ERROR :: %s does not exist!\r\n", p_firmware_location[k]);
   return -1;
  }/*if */

  {
   char firmware_absolute_location[PATH_MAX];
   char *p;

   p = realpath(p_firmware_location[k], &firmware_absolute_location[0]);
   if(NULL == p)
   {
    printf("realpath error :: %s\r\n", strerror(errno));
    return -1;
   }/*if*/

   printf("firmware%d : %s\r\n", k + 1, firmware_absolute_location);
  }
  //printf("firmware%d size = %3.1f KB\r\n", k + 1, fw_dir_st.st_size/1024.0);
  
 }/*local variable*/


 printf("\r\nip address of this server:\r\n");
 {
  struct ifaddrs *ifaddr, *ifa;
  int s;
  char host[NI_MAXHOST];

  if (getifaddrs(&ifaddr) == -1)
  {
   perror("getifaddrs");
   exit(EXIT_FAILURE);
  }

  for (ifa = ifaddr; ifa != NULL; ifa = ifa->ifa_next)
  {
   if (ifa->ifa_addr == NULL)
    continue;

   s = getnameinfo(ifa->ifa_addr, sizeof(struct sockaddr_in), 
     &host[0], NI_MAXHOST, NULL, 0, NI_NUMERICHOST);

   if( AF_INET == ifa->ifa_addr->sa_family)
   {
    if (s != 0)
    {
     printf("getnameinfo() failed: %s\n", gai_strerror(s));
     exit(EXIT_FAILURE);
    }
    //printf("\tInterface : <%s>\n",ifa->ifa_name );
    //printf("\t  Address : <%s>\n", host);
    printf("\t<%s> : %s\r\n", ifa->ifa_name, &host[0]);
   }
  }/*for ifa*/

  freeifaddrs(ifaddr);
 }/*local variabes*/
 printf("\r\n");
 esp8266_ota_server_start(&p_firmware_location[0]);

 return 0;
}/*main*/

And Build the dfu-ota server :

gcc dfu_ota_server.c -o dfu_ota_server


三. Modify your working project and call the function int  device_firmware_upgrade_over_the_air(uint8_t *p_server_ip) while it is time for DFU-OTA.
   Build the binary twice,  one for user1 (the common binary for being downloaded to ESP8266 via USB-UART), one for user2. (while the scrip gen_misc.sh has been executed, "STEP 2: choose bin generate" would determine this build is for user1 or user2), Once the user1 has been build, you should clean the whole output, especially the .o files, before building user2.

 四. Move the built binaries, user1.1024.new.2.bin and user2.1024.new.2.bin to your OTA server and run the dfu_ota_server commnad:

./dfu_ota_server -fw1 user1.1024.new.2.bin -fw2 user2.1024.new.2.bin


五. Before you download the user1 binary to ESP8266 via UART (USB),  It is necessary to ERASE the whole flash. In linux it is very easy, 


sudo python ~/.local/lib/python2.7/site-packages/esptool.py --port /dev/ttyUSB0 erase_flash

But if you are in Windows, as my knowing, the flashing tool would not erase the flash thoroughly, instead it only cleans the beginning flag. How to erase the flash in Windows authentically I do not know.



It is all, once it is in DFU-OTA, the server would send the Binary to the ESP8266, then ESP8266 would write the binary to the non-current working zone. If the CRC-checking is passed, ESP8266 would reboot from the other zone which contain the binary just downloaded.


※ In my case, I assume the flash size of the ESP8266 device is 1MB, if your device size is greater than it, your downloading firmware command may be need to be modified as :

sudo python ~/.local/lib/python2.7/site-packages/esptool.py --port /dev/ttyUSB0  write_flash --flash_size=1MB 0x00000 ../ESP8266_RTOS_SDK-1.5.X/bin/boot_v1.7.bin 0x01000 upgrade/user1.1024.new.2.bin  0xFE000 ../ESP8266_RTOS_SDK-1.5.X/bin/blank.bin 0xFC000  esp_init_data_default.bin --flash_mode dout


The hexadecimal codes are the addresses where the following binaries should be locates. The address of esp_init_data_default.bin is flashing-size depending.
for the common esp8266 modules:

    0x7c000 for 512 kB, modules like most ESP-01, -03, -07 etc.
    0xfc000 for 1 MB, modules like ESP8285, PSF-A85, some ESP-01, -03 etc.
    0x1fc000 for 2 MB
    0x3fc000 for 4 MB, modules like ESP-12E, NodeMCU devkit 1.0, WeMos D1 mini
    0x7fc000 for 8 MB
    0xffc000 for 16 MB, modules like WeMos D1 mini pro

      The command  --flash_size=1MB  is to compel esptool treating  the esp8266 module acting as 1 MB flash, thus the esp_init_data_default.bin would be written in addrress 0xFC000.

    If you would not like the esptool treat your module as 1MB flash only,  you should fill the correct address of  esp_init_data_default.bin. But that might lead my DFU-OTA code non-workable.

   Addition : esp_init_data_default.bin is for the configuration of esp8266.In which, it is able to set some configuration which is unable be set via APIs, e.g. configure the ADC function to measure the system voltage. More detail about the configuration, please refer to this document,  section 3.9.