Short cut to static IP on HypriotOS using linux

So, go try understand the new flash utility for Hypriot OS. Ignore the blog entries as the wacker has no technical writing skills.

Seems you need only read and understand the FAQ. All you need to know is:

>flash --userdata setup.yaml hypriotos-rpi-v1.10.0.img.zip

With setup.yaml based upon one of the examples from the flash samples folder.

First of all you need flash installed on your linux box, since it does not run on windoze.

The flash utility can be installed on Debian 10 with:

>sudo apt-get install -y pv curl python-pip unzip hdparm

>sudo pip install awscli

>curl -LO https://github.com/hypriot/flash/releases/download/2.4.0/flash

>chmod +x flash

>sudo mv flash /usr/local/bin/flash

You can test it with:

>flash --help

OR, you can do what I eventually did, since it took me hours to find you couldn’t set the static IP on a HypriotOS host with the old tricks no more.

So, I set up for flash utility but when I stuck a previously bombed SD card into my Debian 10 box, the SD card of course was readable.

So, here is your short version to set up a static IP with HypriotOS without having to setup cloud-init and all that jazz.

Download a HypriotOS image zip, say:

https://github.com/hypriot/image-builder-rpi/releases/download/v1.11.4/hypriotos-rpi-v1.11.4.img.zip

Unzip the image and burn to sd card using etcher.

If you have the sd card in your Debian 10 machine, open it using graphical file browser. Open and edit the following file using the file browser’s built in editor:

user-data.yaml

Use the example static.yml file in HypriotOS sample folder and edit up the user-data.yaml file to suit your static IP setup.

Take sd card out of your Debian host and insert into your target RPI.

BOOT FOR THE FIRST TIME and voila! You will find your HypriotOS host on the static IP you set up in the edited user-data.yml file.

Note, the BOOT FOR THE FIRST TIME. That is, download image, edit the config in user-data.yml then BOOT FOR THE FIRST TIME.

Nice accident si?

Message repeats, without the flash utility installed :

• Insert SD card into USB SD card reader.
• Insert USB SD card reader into Linux box.
• Bomb Hypriot onto SD card. Do not remove it. DO NOT BOOT IT!
• Use text editor on Linux box to edit user-data.yml file on SD card in the USB SD card reader. The file is at the top of file system so easy to find when you open up the SD card in a file browser.
• Safe the updates to user-data.yml on the SD card.
• Unmount the USB SD card reader and insert the SD card into your RPI.
• Now you can boot.
• You needn’t have used the static IP example but, assuming you did, you now have a HypriotOS box with static IP.

Oh my, O Pi

Just about had enough of Orange Pi Zeros. My home server stopped serving again.

I tried balenaOS and couldn’t get it up on the network.

I then tried ubuntu server same. I thought I must have plumb forgotten how to set up static addresses.

I relented and went and tried latest armbian, despite the ever presence of that wanker Igor.

Now, funnily armbian faired no better. Yet I would not have gotten that wrong since armbian comes with nmtui which handles the fiddly bit behind the scenes.

Go figure. Found a but in redhat against x86 that had the same behaviour. Not much help but it did provide an idea on how to potentially get the system up. The bug required one boot up without Ethernet attached.

So I tried that and voila! Up comes the OPI on static address. Well I felt sure then a bug in all three operating systems running on OPI right? Of course not.

Yep, you guessed it. The OPI stayed up for a while then went offline again.

I had the 3wire serial in so had an avenue to poke around.

I tried shutting down, pulling Ethernet, powering up and connecting Ethernet once booted. I knew in my heart that would not fix it and it hadn’t. The second boot did not see Ethernet come up.

In fact, while the Ethernet was up, I did manage to get snap installed. Snapped in mosquitto and then most of the way through node-red when the Ethernet dropped out.

So, I started poking through the logs and it kept complaining about a loss of carrier. I note also that without doing anything else the network came back up for a while, then went down again. I am suspecting flaky SOC or board. Much like my first experience with OPI.

So, dumping the OPI Zero server for the house and going for HypriotOS on my RPI 3B+.

Pegaleg

So, one file I had fun getting off my laptop was the source for the parser for the google calendar event titles.

The titles, as discussed previously, would be a list of sprinkler group or unit callouts.

I managed to ssh onto the laptop from my desktop to lift anything I wanted before I trashed the laptop with a clean Debian install.

While waiting for the install, I opted to tart up the parser with some sanity.

The problem for the distributed opensprinklette design is that there is no editor per se to police the entries in the google calendar event title. So, if any unwanted text is in the title, the parser running on the node-red server will drop the lot. Harsh I know, but the distributed nature of this entails some trade-offs.

So, I did add some smarts to avoid sending repeated call-outs to a sprinkler. A burst of call-outs will go out once the event title is parsed. The ESP8266 gadgets don’t mind if a single sprinkler is pummelled – it simply results in the time starting from the last received call-out – which won’t be that many milliseconds behind the first.

Still, to opt for a defensive scheme meant testing for a call-out in the return from the scan of the line before adding it to the return. Simples tick!

So, even if the user pummels the title of the event with multiple call-outs to the same sprinkler, the parsed response will only contain a single call-out.

So something like this in the google calendar title:

group1=1;group1=[1];group1=[2,2,2,4,4,4,4,1,1,1,3]

will return the following in the node-red response (prior to fully concatenating into a MQTT topic):
[
"group1/1",
"group1/2",
"group1/4",
"group1/3"
]

Slowly

So, 4 days ago I reported to node-red group that the –beta snap of node-red did not start as a service on armf version – at least when snapped onto Ubuntu Core 16 running on Orange Pi Zero.

When I started this journey, the following command would not download a snap but it would prompt:

sudo snap install node-red

The prompt would report there is no –stable version and I had to choose between –beta or –edge.

In fact, the snapcraft.io site also confirmed this as it only had a beta or edge version in the drop down for armf.

So I would install node-red with:

sudo snap install --beta node-red

But, as I mentioned, it did not start a node-red editor on :1880.

If I used the following:

snap services

It would report the mosquitto snap I snapped in was indeed running a service.

Et voila! If I typed the following:

sudo snap run node-red

Then node-red would run in foreground.

Not much use since it needed to run as a service.

Starting node-red as a service is easy peasy, if you read the help file, with:

sudo snap start node-red

Except, that raised the error “node-red ain’t got no stinking services to run” which confirmed that the –beta snap for node-red, at least the armf version, had something funky going on.

Curiously, on my freshly built laptop running amd64 version of debian 9 stretch, with snapd installed, the snap of –beta node-red ran as service immediately after install.

That points definitely to something funky going on in the armf Ubuntu Core 16 OPiZ combination.

Then, as likely because of my prompting, node-red community pushed a –stable version of node-red out to snapcraft.io (something like 4 hours ago as I type).   The youngest version, –edge, was 6 months ago.

So node-red didn’t have a –stable for armf!!  Some problem with their build (apparently).

Working with node-red community now to sort this.  Such a good Citizen I am.

 

 

It’s a form of self flaggelation …

… this snap stuff.

What I have worked out so far with orangepi zero, Ubuntu Core 16 and Snap.

Ubuntu Core, because they force updates, is failing to come back after scheduled reboot. Or so it seems.

The serial port responds but the device is not on lan.

All attempts to attach at the IP address reported in the message displayed on the serial port fail.

Hard reset (power off, power on) brings board back up with new IP address but then the cycle repeats. Schedule reboot does not come back up.

Weird but if I use a sudo reboot the device comes back up with IP intact.

So, something about the core auto reboot.

I tried a debian server but snapd would not install. Rasbian apparently is a no go, so maybe they meant arm generally?

Otherwise, moquitto snaps and runs fine. The node-red snap installs but no service starts. No config comes with snap and snap start node-red returns no service available message. The node-red snap will otherwise run in foreground.

Snappy!

So, yes.  I have been playing with Snap while doing the Docker course.

I did install Ubuntu Core on the OPi and managed to get Mosquitto snap running.  But the node-red snap keeps failing to connect or install (with time-outs reported).   It is 6 am on Saturday, so is that indicative of northern hemisphere loading on the Net?  Who knows, wife, sister in law and two of there friends just finished their party.  I was woken when the retrobates left.  So, I am dabbling.

The same old story though.

Help is out of date.  I installed Docker version of snapcraft on my linux laptop.  You were supposed to use that if you weren’t using Ubuntu – I prefer Debian.

There wasn’t enough data to get something running on that Dockerized snapcraft.

I then found another help site that pointed out that the Docker version of snapcraft was deprecated.

You can now snap snapcraft into place.

Then found an interesting snap that I subsequently deleted.  It, however, reported an error on start up.  No help on the snap so dumped its arse.

I will perceiver with snapping MQTT and Node-Red onto the OPi.  If that holds its water I might still use the OPi as the house server.

There is, however,  thinger.  There is a RaspingdoodleburryPie image.   I have loaded the snap of the server onto my linux laptop to do a trade-off against Node-Red.

Wait … wait … wait … finally, Node-Red snapped into place on OPi.

So, with Mosquitto running, Node-Red MQTT nodes talking, I have a 5 minute timer to leave the system running to see if it keeps going.

The thing I will need sort, however, is that the snap system will push updates (I could one happening the other day).  So, I will need look into holding off the pushed updates to avoid the updates confounding the testing.  As you’ll recall the initial suspicion was flaky hardware.

Ta da!

Ah ha!

I had some trouble importing my pegjs parser function into the node console until I worked out node and/or javascript treats directories as “/” whereas windows uses “\”.

So finally the import is simply:

var parser=require('./parser.js');

I also hand modified the parse function signature which is now:

function peg$parse(input,zones, options) {};

I worked out that zones needed to go ahead of options as calling “parse(input,,zones)” wasn’t a popular choice with the javascript.

The input variable “zones” will come from the configuration node.  That way the parser will take the configuration object [{sector1:chipid()}] and return the mqtt topic fragment “chipid()/zone[0..4]”.

So, the modified parser tested using node console shows it working:

Now all is needed is the use the zone “grammered” string in the google event title and snip it out, when the event starts/stops.  This is achieved with msg.payload.title as the first parameter in the call made by the node reacting to the calendar event.

The array returned can either be iterated over using a for-loop OR be returned by a node-red function to be treated as a stream of messages (aka msg).

I have a sneaking suspicion that, in fact (and having read the code and how it uses “options”), I [could|should] have injected the zones via options.

Something to play with I think, as it doesn’t seem clean to hand mod the parser code.

Sizzzzle!

Pop!

A parcel turned up with RaspingbreathburryDoodlePi heatsink kits.

They are the recommended source for a heatsink for the OPiZ.

Although I can certainly find the same on Aliexpress, I bought locally to get them within a week instead of 10 weeks.

Still, gotta say brrrrrrrrrrrrrr, freezy!

At least 5degreeC cooler running with heat sink.

Still not convinced that was source of ethernet dropouts on my new board.   No problems for a couple of weeks now so I will put concerns aside.

I did notice 15mmx15mmx5mm 5V fans on Aliexpress though.  Tempting.

Still tempted at getting a small peltier cooling gadget to go the whole hog – mostly for the fun of it.

But, at the moment, I am focusing on finishing the opensprinklette configurator node for in node-red.   I have a basic configurator to map sectors to WeMOS chipid(), now I will integrate the pegjs defined parser.

Small steps

I have roughed a node-red config node and it’s visible counterpart.  I decided to call wemos nodes “sectors” – since the usual rort is to call a single channel on an irrigation controller, controlling a single solenoid, a zone.

I have set up for 4 sectors each using a WeMOS D1R2 with a quad relay shield.  That provides up to 16 zones (4 per WeMOS).

You need to mod the relay shield with a couple of pullup resisters.  This is to get around a design shortfall on the WeMOD D1R2.

I am using a protoshield in between the WeMOS and the relay shield to allow for fidgeting with design changes. I added a four position dip to set the zone id BUT I dropped that in favour of the config node concept.

That gives me back 4 pins for GPIO. The problem is the out and out lie that the WeMOS is a Uno form factor. The ESP8266 has to cheat by using the same GPIO pins across a couple of Arduino socket pins.

I will add the rain gauge input later.

The idea is an input line such as the following as the title of the google calendar event:

sector1=1,sector3=4,sector2=0,sector3=1

The line above shows you some of the input features I will aim for, being:

• You don’t need to nominate all four sectors or even all four individual zones of a sector.
• You can order the sectors in any order.
• The zones per sector are number 0..4 with 0 being the global all on/off id for all 4 zones on the addressed sector.

I did think about using JSON as input but the problem is that if you have two tokens the same then the object construction takes the later value in the line for the key.  Oh well.

For this to work you attach two google calendar event sniffers, one to flag the start of an event and one to flag the end.  Both feed into the opensprinklette-configurator to decode the events into MQTT calls to the 4 WeMOS D1 R2.

Of note, the configurator maps between chipid() and sector# (and back again) so planning the sprinkling can be in people-talk (relatively speaking).  At least you don’t have to remember which WeMOS chipid() was allocated to what sector.

Of course, there are quirks to do with the distributed system.  There will be a watchdog on the WeMOS to automatically turn off the water to a zone after 45min.  To plan out longer watering you will need back to back calendar events (shorter than 45min to avoid the watchdog).  If that is offensive please send money to help me pay the water bill of a runaway commercial setup ($3,000 in fact).

The other option I guess is send the duration to the WeMOS node and let it do the countdown.

I will play with a couple of timing approaches to see what is most robust – given you could have the ISP gateway drop out, the node-red crash, the emqttd crash, the OPiZ drop of the network.  Not to mention, google calendar hickups – I occasionally get a baulk around credentials lapsing that somehow comes good again.  Oh and of course, the WeMOS could also behave badly.

All in all needs a good bashing to help weed out nuisances.

Back to work

So, at last, now that the OPiZ setup saga is over (fingers crossed) we begin over.

I built a new nodemcu firmware for the WeMOS to include:

• bit
• end user setup
• file
• GPIO
• MQTT
• net
• node
• RTC Time
• SNTP
• timer
• UART
• Wifi

A few coding snippets later and  the WeMOS can catch the mqtt topic running on the OPiZ feed by the node-red on the OPiZ.

So, I can now parcel up the OPiZ as the house server and tidy up the sprinkler system.

I did note that the WeMOS did not come up first time I bombed the firmware.  That was sorted (it seemed) by selection the 4MB Flash option on ESP8266Flasher.  The ESP-12E on the WeMOS has a 4MB flash and I noted that there is a branch of the nodemcu frozen in time now for the 512kB chips – so go with the master branch if you have the WeMOS D1 R2.

I did also manage to break a hoodoo now that nodemcu does away with autoconnect for the MQTT.  The timer callback scheme works a treat.  I can reboot the OPiZ and the WeMOS will reconnect once emqttd is up and running again.  Of course, I have a LWT setup so that the emqttd server will tell the node-red if the WeMOS drops off the channel.

This is getting exciting now.