So over the weekend, I had a chance to work on this a bit more, and I think it's getting close to fully stable, and is already usable aside from the fact that there's no standalone Python library that isn't attached to a web UI, but the Arduino libraries are all ready to go.
Since this was my Epic Quarantine Project, and it's working(and in limited non critical use at work), I thought it would be worth doing a (long and not well formatted...) writeup here in case any of this development is useful.
The goal here is one protocol that can handle ultra low power portable use, one-hop mesh, and typical smarthome products, and discourages products that use it from ever becoming e-waste. It's meant to support bluetooth style plug and play use, not require any boilerplate and manual work configuring registers and such to use, to be really cheap, and not to waste bandwidth any more than neccesary, with auto-TX power as a baseline standard.
What makes this a little different from the others is collision-tolerant addressing and error correction. It uses Arduino and RFM69 modules(I *think* CC1101 works as well), in the usual FSK mode, but adds a layer of Golay code error correction for added interference resistance.
It's obviously nowhere near LoRa's range and performance, but it seems to go 20-30% farther than FSK without error correction, and it seems to have much lower packet loss out towards the edge of coverage, it's faster than LoRa, and honestly, I'm not sure the airwaves can handle a million people with cheap LoRa key trackers....
Devices exist on a "virtual channel", which is defined by a 32 byte key that encrypts all traffic. Within a channel there are 255 node IDs available, but some things are designed mostly assuming you'll only have one device like a sensor, plus a hub.
The packets themselves only contain a 20-bit hash of the channel and the node ID(which is plaintext, it's part of the IV to prevent key reuse). If there's a collision, that's fine, you just ignore it if it doesn't decode correctly.
In addition, you can also use a 20-bit "Private hint sequence" instead. This is a rolling code that changes every 16 seconds. The library doesn't actually do this, but I may add a flag to do so. In theory, this makes tracking a lot harder. There's no MAC address or other fixed value to sniff, besides the node ID, which is one byte and a lot less interesting. You do need to exchange normal packets to sync the time, but not very often.
The protocol is fairly replay attack resistant, we discard any packet older than the latest we have seen on the channel, and the only way to set the clock backwards or jump is through challenge-response authentication. To the user, it's completely connectionless and clock sync happens in the background, you just set the keys and start sending and listening.
There's some weaknesses, and I wouldn't call it bank grade, there's no session keys or anything, so there's a small chance of a reused key slipping in somewhere if you don't have an RTC, since the IVs are only 7 byte timestamps plus the ID, but I think it may be secure enough for what it's meant for.
On the infrastructure side, there is an Arduino sketch that can act as a gateway interface for a raspberry pi. The code on the Pi side doesn't exist as a standalone library, but there is support in my other project Kaithem, which is open, and I might make a standalone version eventually.
To compensate for the short range of plain FSK, there's support for using multiple gateways. Clients talk to a gateway via MQTT, and you can stack as many as you want for transmit and receive diversity, automatically throwing away duplicate packets, and automatically choosing the best gateway to send out of and the best power level.
The encryption keys do need to be sent plaintext via MQTT, because the Arduino itself does most of the decoding work, and also to keep things simple and give channels one single easy to work with ID, the key.
For low power work, we reuse the rolling code feature of the private hint sequence, and have beacon frames that don't have any data in them. Because of the rolling code, you can't really forge the presence of the device beaconing(Although you can forge the reply), so electronic leash type tethers can be done simply.
To wake the device up, we have yet another type of beacon frame, which uses a salted hash of the channel key. It is short and doesn't need any decryption, so we don't have to stay awake too long to receive it, and it's a rolling code, so there's no way to do a battery rundown attack without knowing the key.
And finally, for plug-and-play goodness, we have a "pairing" feature, which isn't so much a pairing as a way to program a device over the air with arbitrary RF parameters and channel keys(Using elliptic curves to deter sniffing the key during pairing), and a "structured message" type.
Structured messages are sets type-channel-length-value records(channel is used for things like multiple sensors of one type), that can carry up to 8 bytes, and only need a 2 byte header per record.
Types under 192 are reserved, so there's a lot of room for sensors and devices that "just work". The base firmware only does one thing with these, and that's config data, a simple region of memory you can read, write, or save as default to EEPROM. There's no actual reserved meaning or anything to any of the config bytes, the intent is that you can either make a programmer for a specific device, or just publish some "cheat codes" for more advanced users to copy and paste to tweak things.
This does use about 80% of an Arduino's flash, but I'm not too concerned. It's meant for very simple things, or for better processors like the Feather M0, and there's still plenty of space left for a temp sensor and some switches, especially with almost all the communication work done for you. https://github.com/EternityForest/SG1