Workflow for Micropython development

on

 There are many workflows for Micropython development:

  • If you have just one program file, use "mpremote run main.py"
  • You can use "mount . run main.py" to mount your current directory on the PC on the microcontroller, and copy no files at all to the device. This is an excellent option, see the mpremote docs.
  • If you prefer an IDE, there is Thonny, Pycharm, and others
  • For editing .py files, there are many, many text editors, you even can use old and faithful vi
  • To copy your files to the microcontroller. mpremote, pyboard and many other options
  • You can use rsync or mpbridge and other utilities to synchronize folders on the PC with the microcontroller.
I just want to explain here what I have done.

I use Windows, but this will also work on Mac. I use a text editor (Notepad++) that I like. I use mpremote to copy files to the microcontroller (currently version 0.40.0), since this is currently the best maintained utility to update the microcontroller.

To copy update files to the microcontroller I use GNU make (free), with a Makefile. Make is a venerable and heavily used utility program for development, if you are not familiar with make, read the documentation and tutorial for GNU make.

The point here is to know which files have been edited or changed, and to copy only these files to the microcontroller. But there is no easy way to compare file date and time with the microcontroller files. My microcontrollers don't even have updated date and time!

So I create a "device" folder that will be then filled by the make utility, to check which files are up to date. For each file to be copied to the Microcontroller, there has to be an entry in the Makefile and a rule. The make rules copy all files to both the microcontroller and to a folder called device to check for timestamps, so only updated files are copied. 

I have two macros, $(COPYFILE) and $(COMPILE), here is an example for an ESP32:

define COPYFILE
    mpremote cp $< :$(subst \,/,$(subst device\,,$@))
    copy $< $@
endef

define COMPILE
    mpy-cross -march=xtensawin $< -o $@
    mpremote cp $@ :$(@F)
endef

esp32 : \
device\main.py \
device\module1.mpy \
device\module2.mpy 

device\main.py: src\main.py
    $(COPYFILE)

device\module1.mpy : src\module1.py
    $(COMPILE)

device\module2.mpy : src\module2.py
    $(COMPILE)
When you run make, the utility will check file dates and times of the files in src and if they are newer than the device folder, it will execute the rules, and either compile and copy, or just copy the file both to the device folder and the microcontroller.

I also have a "install" target in the Makefile. This target should create all folders both on the microcontroller and the device folder, and should install prerrequisite libraries, i.e. with mpremote mip install.

A "clean" target is also useful. I run a Micropython script with mpremote run to delete all files on the device and shell commands to delete the device folder.

I like this workflow:
  • It's fast. Since the .py files are compiled, the size shrinks to about 50% and copying is faster. Only updated files are copied.
  • When I run make, I see again a list of all files that I have changed. 
  • I tend to forget how I setup a new device, but "make install" (or "make clean"/"make install") will do it. For me, this is a major point. I used to have documentation on how I set up the microcontroller, but then I forgot to update the documentation :-(
  • The .py files are all compiled, that's a little bit more efficient at device startup (not a big difference for me, but every advantage counts).
  • It's easy to switch to a new microcontroller.
  • There can be targets for different architectures, if needed.
  • I can keep configuration files or data files unchanged during updates, if I want to.
  • I could have "development" and "production" targets.
  • I am more confortable with the make process than with file synchronization utilites, since I feel that I am controlling what will be copied and what not.
  • Once setup, it's only necessary to update the Makefile when adding a new file to the project.
  • The process is repeatable and stable. This is the main advantage for me.
Drawbacks:
  • You have to install GNU make
  • You have to write a Makefile. Include the macros, and include a rule for each file. This took me a couple of minutes for a project of 50 files (.py, .html, .jpg, .css, .json, files).

Comments

Post a Comment