Archive for the The downside of Opensource Category

HA+ESPHome no good for critical systems

Posted in ESPHome, Home Assistant, OpenSprinklette, RaspingBreathburryDOodlePi, The downside of Opensource, The Downside of software development on May 2, 2020 by asteriondaedalus

Ah, actually, I think I have to go away from using ESPHome and homeassistant combination for OpenSprinklette in any event. 

No longer then interested chasing down a bug I was chasing in the HA REST API and using CURL to turn sprinklers on with a POST to switch services. I was planning to use that to be able to write a Kotlin app for an old Android tablet, since 1) I can’t get HA to open on the old version of Chrome on the tablet and 2) the HA app for Android won’t install on my old Android tablet. The bug I found sits cleanly in the REST API of HA since I am only using the examples and the POST does not work for switches or at least switches implemented using ESPHome.

The problem leading me to drop the bug in the REST API is not associated with the REST API but is because my raspingdoodleburry pi has crashed.  But, I found the ESPHome based sprinkler system has 3 of 4 relays on with the Pi crashed and not online. Even worse, as bringing the Pi and therefore HA back online does not automatically reset ESPHome device. Sure, you could set up something on HA start/restart but that does nothing for the span of time the Pi/HA is down and relays on ESPHome device on. Way too fiddly to sort all likely mishap cases. Especially, if the actual cause of the Pi/HA crash and the breakdown of HA/ESPHome communication causing the relays to come on, is unknown. So, even if the Pi/HA came back up by itself (some time after it crashed), there’d still likely be $K dollar water bills in the mail.

Having had a $3k water bill some time back, due to sprinkler system leak, I gauge it makes no sense keeping this combination of HA and ESPHome for a “mission critical” application, since I am not sure when the p p p p p Pi went d d d d down.

The evidence of a HA/ESPHome interaction problem is the relays in this HA/ESPHome version are set up in automations, to come on and turn off in sequence, with a gap in time between off and on, to avoid having to deal with sharing the water pressure. This works fine with HA up and running. So, having 3 of 4 on will be an artefact of Pi+HA crashing and likely some interaction between HA and ESPHome. That is, there are no automations in HA that would set 3 of 4 relays on at the same time and so no two relays should be on at the same based on the design of the automations. The automations have been running on a test rig and come on sunrise and sunset without a problem and the logs did, till the crash of Pi, reflect that expected behaviour.

That likely means the usual ping ponging between two user groups blaming the “other” software and, frankly, not my bag. I want to water my gardens and not solve a problem with use cases not being tested.

Not a problem, I have previously hand written code to run on the WEMOS UNO format board for the sprinkler system. That code is MQTT driven and was designed from scratch for a sprinkler system with ample safe guards. For example it wont turn on water for less than 5 mins, won’t accept more that a 15 minute request and will turn off a sprinkler after 20 minutes. It will allow a new time setting over top the old so if you want a hour of watering you just ping it with 4×15 minute requests every 15 minutes. I have a node-red setup that also will send an sms if a sprinkler node (either a 4x or 1x wemos setup) raises an LWT. I’ve even designed my own boards to support the 24VAC to WEMOS voltage conversion.

I also have a peg based parser to all setting of sprinkler times, across multiple single and quad sprinklers, via titles of google calendar events. Though, I want to decouple from google and try the same over a local mechanism for both principled and practical reasons.

So, I am also looking at how to hook my parser code into other calendar widgets. I am still to look at HA calendar widgetry, if it exists, since automations are fine but the natural thing is to just want to use even just a weekly calendar and not a full year calendar, or so I imagine I think I want to believe. Although, then node-red calendar widgetry or HA calendar widgetry? It will depend upon ease of integration.

In the end, I actually suspected my hand coded sprinkler option was likely the better approach for exactly what the crashing doodle woodle pi and the interaction between HA and ESPHome has revealed. I would even suspect the ESPHome web site, and the HA one to, have caveats and warnings about not using either for mission critical applications. Especially, if its the emergent properties of the integrations that will get you and no one is really going to arbitrate that except for the end users.

Lucky I found this problem on the test bench.

UPDATE

I poked around a bit with this, the problem is a little more insidious.

I thought the server had crashed as I could not log in with MobaXTerm. In fact, I have my server with Ethernet wire connection to a wifi extender and it turned out that the wifi extender has been dropping connection with the ASDL router. A real pain since I bought a new wifi extender BECAUSE I had the same problem with the older wifi extender AND BECAUSE the “help” on the Net suggested there was problems with the older wifi extender.

So, I found that re-connecting the wifi extender to the ASDL router saw an interesting picture. The ESPHome device must have reconnected to the HA server since the switches on the HA HMI, representing the relays in the ESPHome device, had the same state as the device. 1, 2 and 4 on, 3 off.

Not a problem you say. Well, it is since the way the system is set up is that you turn the switch on at HA HMI, and then an automation runs and turns the switch off after 15 minutes.

On top of that, that behaviour was also required when driving the relays by sunrise/sunset since a delayed trigger to turn the switch on, on sunrise/sunset, was used and the trigger to turn the switch off after 15 mins was expected then to trigger on the switch being turned on. Worked a treat while connection between HA and ESPHome device stayed up.

The problem still remained that three relays on ESPHome device were on (1,2,4) and one was off (3). Three switches on HA HMI were on (1,2,4) and one was off (3). HA log reported all switches off. So, there was a disconnect between states internally to HA somewhere.

The automations for all four switches are all triggered on sunrise/sunset. They have delays that set the individual switches on 20 minutes apart. The second set of automations, triggered by individual switches going on, turns the triggered switch off after 15 mins. So, the system works:

  • sunrise->all four switches triggered with delay x mins.
  • 0 mins->switch 1 on trigger switch 1 off automation
  • 15 mins->switch 1 off by switch 1 off automation
  • 20 mins->switch 2 on trigger switch 2 off automation
  • 35 mins->switch 2 off by switch 2 off automation
  • 40 mins->switch 3 on trigger switch 3 off automation
  • 55 mins->switch 3 off by switch 3 off automation
  • 60 mins->switch 4 on trigger switch 3 off automation
  • 75 mins->switch 4 off by switch 4 off automation

Nothing really then to account for why 3 of 4 relays on, why HMI reflected ESPHome device state and then why if relays are on in ESPHome device the associated switch on in HA did not result in a switch off after 15 mins. That is, I have had the ESPHome device powered up and 3 of 4 relays have been on for two days, where they should have off after 15 mins if triggered on by HA. If I manually switch the switches at HA HMI to off today, they are then off at device. I haven’t rebooted device because I want to see if the states coherence is re-established. I assume that states will be coherent again if the automations run tonight and the relays behave properly again. Noting that that won’t redeem the setup for use in a critical applications, since it would take too much energy to understand the interplay and the problem it is actually a state coherency problem in either or both the design of HA or ESPHome or both.

So, no real way to account for 3 off, for example, since all four switches run with exactly the same automation descriptions. You would expect all four on or all four off, not a mixture.

And here we go again …

Posted in ESPHome, Home Assistant, The downside of Opensource on April 19, 2020 by asteriondaedalus

Just bracing myself for a torrent of UNHELP on homeassistant user group.

The problem, well there is a couple of problems.

Problem 1. I found the charger for my old Acer A500 android tablet so that is being set up in kitchen as web search, YouTube and Netflix gadget for cooking assistance. That means, apparently, I cannot use it for homeassistant client since a) Chrome version on A500 is too old and b) the homeassistant android app is not compatible with the version of Android running on the A500 and therefore c) see Problem 2.

Problem 2. I need to ask questions on the homeassistant user group.

Problem 3. I need to sort why the rest API works fine in all examples provided on homeassistant web site EXCEPT! for the POST that is to set the sprinklers on. When I use the POST it complains their is a non-JSON problem with the data despite the token, IP address, device id all work fine with other API calls and so that means see Problem 2.

I was able to play with the API by installing curl onto my widoze 10 laptop and then GETing and POSTing to my raspingdoodleburry pi home server running homeassistant under hypriot/docker.

Why all the bother? I will likely write a Kotlin app, for my old A500, with n buttons and POST calls to sprinkler zones to the ESP Home device via HA since HA has been set up with default sprinkler times (so you just turn the sprinkler on).

I can go to a fallback and send MQTT messages if I must, but at the moment the preference is to try to use the API. Generating a POST message in Kotlin is straight forward. The fun bit might be reading back states to make sure the sprinklers are up (so ghosting or not the buttons) and changing button colour or toggling the button on Android device by following the switch state, so as to provide an indication of ACK from device.

Putting that raspingdoodleberry Pi to good use

Posted in Docker, Raspberry Pirate, RaspingBreathburryDOodlePi, The downside of Opensource on December 22, 2019 by asteriondaedalus

So, used Etcher to burn an SD with HypriotOS at:

  • Version 1.11.4
  • hypriotos-rpi-v1.11.4.img
  • Docker 19.03.4
  • kernel 4.19.75
  • Dated 04.11.2019

Booted the RPI 3B+ and added docker-compose via:

  • sudo apt-get update
  • sudo apt-get install -y apt-transport-https
  • echo "deb https://packagecloud.io/Hypriot/Schatzkiste/debian/ jessie main" | sudo tee /etc/apt/sources.list.d/hypriot.list
  • sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 37BBEE3F7AD95B3F
  • sudo apt-get update
  • sudo apt-get install docker-compose

Grabbed my favorite MQTT setup, all grown up now, emqx with:

docker pull emqx/emqx:v3.2.7

Thence onto node-red with:

docker pull nodered/node-red

The fiddly bit seems to be to get the MQTT nodes in the node-red to connect to the emqx broker. None of the standard tricks seem to work, such as the default use of –LINK from the node-red guide on using it under docker. Hence, the downloading of docker-compose as I want to see if starting emqx and node-red via compose sorts the problem.

The problem is eclipse-mosquitto nor two or three other MQTT brokers, including emqx never see a connection from the node-red container. I tried openning up ports etc. I opted for emqx since it comes with an admin console so it is the most straight forward way to 1) see that the MQTT broker is up and 2) you have visibility of client connections if/when they are made.

Otherwise I am likely to recommend the MobaXterm as it allows me multiple sessions and session types. I can ssh into the hypriotOS or VNC onto my debian server at the same time.

I also opted for MS Visual Studio Code, since I use is for my PlatformIO dabblings. The remote explorer allows me to edit files on the RPI/OPI/BBB/C.H.I.P/ODRIOD W/ODRIOD C1/Parallella/NVIDIA Jetson Nano.

What a wanK!

Posted in hass(le)io, The downside of Opensource on July 15, 2019 by asteriondaedalus

So, to date. Things that don’t seem to work out of box on Raspberry Pi 3B+ hass(le)io (either 32 or 64 bit) are:

  • cloud9 ide
  • node-red
  • portainer
  • a few others that I can’t be bothered listing

Now the problem might be mine. It seems though that the setting up of the config files is trickier than it should be. This is aided and abetted by “help” files that don’t.

Idiomatically, a templated warning on many “help” files is “Don’t use this config file, write your own”. The example config in the “help” file acting, as it were, as taunt.

The problem turns out, if you cut and paste the example config files, from the “help”, many of them cause errors! They don’t, therefore, act as a example of a straight forward, no bells’n’whistles, get you started type of example.

What seems to be missing, in peoples puny attempts at “help”, is:

  1. Help should not assume users have full architect status in the project, and understand not everyone has the priori knowledge of an architect OR even the plug-in developer.
  2. See point 1.

Certainly, when some of the error logs are reporting potentially missing files, nested deep in the Docker image, then the fact that the Docker image is not ready for production, and shouldn’t be on the streets, is laid bare.

Some of this is the open source, the user is the tester. Or worse, the user is a developer. I know that there is a derogatory name, in the open source community, for users who don’t contribute. I admit I am one. What I marvel at is the pressure to contribute is a counter argument to the utility of open source, since if I can’t use the software due to useability, installability, understandability, portability or other software quality problems then I am not likely to be much help because of the inherent problems, generally, with open source in the areas of useability, installability, understandability, portability.

That is, if using the software is my goal, getting involved in all the open source code bases I might opt to use for my projects is not an aspiration, BECAUSE I am officially bored with the quality of open source software in terms of useability, installability, understandability, portability.

You should be too. Especially now that open source projects are whining over “pay me for my time”. YOU FREAKS! You were the types that set up the free software rort. You would have been passing floppies of stolen software around, putting companies out of business. SUCK IT UP YOU PRINCESSES!

Pardon my grumpy pants. I will actually buy a beer, if the option is there, if the software meets my expectations on useability, installability, understandability, portability. Not, however, if in the first hour I am having to debug what should be straight forward, or clearly explained.

I did managed to get the Aircast running, no changes to config needed other than, it seems, turning SSL flag to false. That was a punt I took, as the default (SSL=true) raise a error when you tried the web UI link. That “fix”, applied to the other apps above, did not fix them.

Aircast is great, since I have a C.H.I.P. running shairpoint-sync under the television. So, as chrome-cast is attached to television, I can free up a airplay speaker.

The other app that seems to run fine is motioneye, which is a CCTV monitoring thing. Same thing, came up after setting SSL=false. CCTV around the place was my next project for the home, so sweet.

The configurator works, so at least I can play with the scripting.

Otherwise, I am running the hass(le)io based RPI3B+ side by side my OPiZ.

The OPiZ is running Mosquito and node-red just fine, except for the system going down about once a month.

Psyched!

Posted in Rant, The downside of Opensource on March 20, 2019 by asteriondaedalus

So, OpenMV M7 from ebay.  Watchout!

The price compared well with the OpenMV store, which reported out of stock on M7.

So, I bought of ebay.

The problem, it appears there is a need to register the boards with OpenMV to get access to the OpenMV IDE. 

The board from ebay is registerable BUT it costs $15 per board, in addition to the purchase price, if it doesn’t come registered via OpenMV.

I otherwise assume that the boards from OpenMV come pre-registered, since the registration make sense.  It makes sense because open hardware is a way to no make money, since you give up your IP and it then gets copied.  Adding board registration, though not really open sourcy, seems like a good way to make sure the software side stays viable. 

Sure, copy and sell the boards.  You don’t have to pay a licence BUT the user buying your board does.  So, why charge the same as a registered board from OpenMV?

Rip off?

Not really, the OpenMV M7 is USD$65, I paid AUD$61 then AUD$21 to register so that’s AUD$82 which is AUD$10 cheaper than the USD$65.

The scary part being what, if like FDTI chips, you couldn’t register clones. Now that would be saucy.

Crap!

Posted in Rant, The downside of Opensource on November 11, 2018 by asteriondaedalus

So, found I couldn’t run tensorflow on my laptop as the older i5 I am running is missing an instruction extension to support vectorisations.

I thought then to try to get cuda toolkit up so I could use pyCuda or Cuda accelerated pyTorch.

I got the nvidia driver installed for the GeForce GT 420M my laptop was running. Tricky since I am running Debian 9 Stretch and nvidia is anti-Debian.

Tried unsuccessfully to get pyCuda to talk to cuda toolkit.

There is no official support for the kit on Debian so I had to rely upon vain attempts by others.

I eventually found a good blurb on the problem, but my machine started blurting out app crash reports from the desktop. They kept coming through as often as I closed them. I decided to reboot.

DOH!

Laptop came up without desktop running.

A little poking round and I worked out that sddm was not running and would not run. So, no sddm, no x-windows, no desktop. Kulprit appeared to be the nvidia supplied driver for the 420M.

I tried uninstalling the nvidia driver but the driver uninstall defaults to not recovering your x-windows setup. Unfortunately, I must have twitched on the enter key so nvidia uninstall wiped itself clean, except for the x-windows config.

I poked around and found one config file that needed to have nvidia changed to nouveau BUT there was no information on the extent of mods to files required to manually backout of the nvidia driver callouts in the x-windows.

I was stuck with no desktop.

I also found I was now also sans wifi and likely few other things so I elected to rebuild the laptop (again). That is the standard approach to dealing with linux is it not. If only that was not such a drag.

He’s baaa … ack!

Posted in Orange Pi, The downside of Opensource, Ubuntu Core 16 on September 9, 2018 by asteriondaedalus

My Armbian troll, the moderator on Armbian who banned me from Armbian forum because I pointed out he was off topic and providing “interesting” but unrelated factoids, has popped up on Orange PI forum.  My Armbian troll has tagged my observations about the lack of readiness of Ubuntu CORE 16 and snap generally on Orange Pi Zero.

517283778-612x612

 

 

 

Fracking wanking open source user groups

Posted in The downside of Opensource, Ubuntu Core 16 on September 7, 2018 by asteriondaedalus

Well not really open source. It’s Ubuntu.

Ubuntu that commercial Debian front trying to steal back money having jumped on the “software should be free” band wagon.

So why are they operating like open source with dopey “user” forums?

Costs nothing right!  Worth as much.

Same old same old.

No “help”.  No bug reporting.  Everything must be  “question”.

My questions are not answered.  My questions earn, or not, “reputation”.  But they are not answered.

Over 100,000 unanswered questions on Ubuntu forums alone.

Many with “reputation” because they are, what? interesting?   But not answered.

My questions: Why is Ubuntu handing out images of CORE 16 that go into tight useless config prompt loops after first forced update?  Why am I having to go through this inane user group crap to “help” them sort their frackup?

Its as much as what I have seen in virus checkers when they frack up.

The user forums obfuscate the threat to the vendor reputation because the vendor has users in set-two mode, turning on one another for “reputation”, so you cannot gauge the impact to the community of the frack up.

So.  As usual if I answer my own questions the answers are immediately denounced as not answers.  They are not answers, of course, as to the literal niggly technical reason that the Ubuntu CORE 16 is fragged in the Orange Pi Zero distro.   That is, whether a flag was or wasn’t set just so.

The answer, however, is that the distro was released without proper testing.  That flag, you wankers, was supposed to be just so.

Ubuntu Core 16 on Orange Pi Zero: minus 1 billion reputation points.

Slowly

Posted in MQTT, node-red, Orange Pi, The downside of Opensource, Ubuntu Core 16 on September 6, 2018 by asteriondaedalus

So, 4 days ago I reported to node-red group that the –beta snap of node-red did not start as a service on armf version – at least when snapped onto Ubuntu Core 16 running on Orange Pi Zero.

When I started this journey, the following command would not download a snap but it would prompt:

sudo snap install node-red

The prompt would report there is no –stable version and I had to choose between –beta or –edge.

In fact, the snapcraft.io site also confirmed this as it only had a beta or edge version in the drop down for armf.

So I would install node-red with:

sudo snap install --beta node-red

But, as I mentioned, it did not start a node-red editor on :1880.

If I used the following:

snap services

It would report the mosquitto snap I snapped in was indeed running a service.

Et voila! If I typed the following:

sudo snap run node-red

Then node-red would run in foreground.

Not much use since it needed to run as a service.

Starting node-red as a service is easy peasy, if you read the help file, with:

sudo snap start node-red

Except, that raised the error “node-red ain’t got no stinking services to run” which confirmed that the –beta snap for node-red, at least the armf version, had something funky going on.

Curiously, on my freshly built laptop running amd64 version of debian 9 stretch, with snapd installed, the snap of –beta node-red ran as service immediately after install.

That points definitely to something funky going on in the armf Ubuntu Core 16 OPiZ combination.

Then, as likely because of my prompting, node-red community pushed a –stable version of node-red out to snapcraft.io (something like 4 hours ago as I type).   The youngest version, –edge, was 6 months ago.

So node-red didn’t have a –stable for armf!!  Some problem with their build (apparently).

Working with node-red community now to sort this.  Such a good Citizen I am.

gc

 

 

It’s a form of self flaggelation …

Posted in MQTT, node-red, Orange Pi, The downside of Opensource, thingbox, Ubuntu Core 16 on September 6, 2018 by asteriondaedalus

… this snap stuff.

What I have worked out so far with orangepi zero, Ubuntu Core 16 and Snap.

Ubuntu Core, because they force updates, is failing to come back after scheduled reboot. Or so it seems.

The serial port responds but the device is not on lan.

All attempts to attach at the IP address reported in the message displayed on the serial port fail.

Hard reset (power off, power on) brings board back up with new IP address but then the cycle repeats. Schedule reboot does not come back up.

Weird but if I use a sudo reboot the device comes back up with IP intact.

So, something about the core auto reboot.

I tried a debian server but snapd would not install. Rasbian apparently is a no go, so maybe they meant arm generally?

Otherwise, moquitto snaps and runs fine. The node-red snap installs but no service starts. No config comes with snap and snap start node-red returns no service available message. The node-red snap will otherwise run in foreground.