topaLE
/
ahoy
mirror of https://github.com/lumapu/ahoy.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
49 Commits
6 Branches
14 Tags
44 MiB
 
 
 
 
 
 
Tree: 97dfbb5936
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '97dfbb5936'
${ noResults }
ahoy/doc/hoymiles-format-description...

560 lines
26 KiB
Raw Blame History

About this Document
===================
This description aims to document the data format that Hoymiles
micro inverters use to communicate their current operating state.
The original Hoymiles setup requires connectivity to "the cloud",
see [this section below](#system-description).
With the information documented here, it is possible to interact with
a set of Hoymiles micro inverters in a purely "offline" way, i.e.
without requiring internet access or any connectivity to a "cloud".
The only required hardware is a Nordic "NRF24L01+" wireless module.
The `ahoy` project at [5] collects software for various platforms,
including Aduino and RaspberryPi.
> Note: Some of the sections in this document are (still) in German. Translations
> may be provided if and when necessary.
Origin, Contributors
--------------------
The information in this document was gathered in a large community
effort which started out with [this post][1]
on the German [mikrocontroller.net][2] forum.
As of April 2022, this effort is still ongoing. Not all details have
been documented yet, and not all secrets have been uncovered.
Multiple members of the community have already successfully retrieved (and
continue to successfully retrieve) data from their Hoymiles micro inverters.
Here's a list of some of the early contributors:
- sorbit: created the original mikrocontroller.net thread
- Martin (Gast): DTU and RF analysis
- Hubi: protocol analysis
- Marcel: initial analysis and much logging and interpretation
- Pascal A. (pasarn): various datagram fields, crc8
- Frank H. (fh_): discovered time_t
- Thomas B. (tbnobody): protocol analysis, logging
- Arnaldo G. (arnaldo_g): data capturing
- Oliver F (of22): protocol analysis, logging
- Martin G. (petersilie): protocol analysis, logging, RaspberryPi
System Description
==================
Ein Setup wie von Hoymiles vorgesehen, sieht wie folgt aus:
- Eine "DTU" kommuniziert mit vielen Wechselrichtern.
- Die Kommunikation geht immer von der DTU aus:
DTU stellt Anfrage und erwartet eine Antwort vom WR.
- Dafür muss die DTU die Adressen (=Seriennummern) aller WR kennen.
- Diese werden der DTU im Rahmen eines Einrichtungsprozesses beigebracht.
```
Nordic
"Enh. Shockburst"
2.4 GHz
\|/ <-----------------> \|/
| |
+-------+ +-----------+
| DTU | | MI-600 |
+-------+ +-----------+-+
| MI-1500 |
+-----------+-+
| MI-... |
+-----------+
:
:
ABBILDUNG 1: Systemübersicht
```
```
Nordic
WLAN "Enh. Shockburst"
2.4 GHz
\|/ \|/
| |
+---------+ +-----------+
| ESP8266 | | NRF24LE1E |
+---------+ +-----------+
^ ^
| |
| +----------+ |
+-----> | GD32F303 | <-----+
(B) +----------+ (C)
ABBILDUNG 2: Innerer Aufbau "DTU"
```
```
Nordic
"Enh. Shockburst"
NRF24LE1E 2.4 GHz
+------------------+ \|/
+----------+ | | | |
| GD32F303 | <----->| µC | NRF24L01+ |-------+
+----------+ (C) | | |
+------+-----------+
ABBILDUNG 3: Detailansicht GD32F303 - NRF24LE1E
```
Adressierung
============
Die Seriennummern der DTU und der WR werden wie folgt als Adressen für die
Kommunikation verwendet:
**Interne Kommunikation**: Die meisten Datenpakete enthalten Quell- und
Zieladresse der jeweiligen Gesprächspartner. Hier werden 4-Byte-Adressen
verwendet, die direkt aus den letzten 8 Stellen der Seriennummer des
Wechselrichters bzw. der DTU gewonnen werden:
Beispiel: Seriennummer ....72818832
Innerhalb der Pakete auf (C) wird daraus die 4-Byte-Adresse
0x72, 0x81, 0x88, 0x32 gebildet. Das ist die BCD-Darstellung
der letzen 8 Dezimalziffern.
**NRF24 addressing scheme**: Over the air, the inverters communicate using
the [Nordic "Enhanced Shockburst" Protocol][3] configured for
5-byte addresses.
The inverter serial number is converted into a "Shockburst" address
as follows:
- encode the final 8 digits of the serial number in BCD format:
`0x72, 0x81, 0x88, 0x32`
- reverse the order of the bytes:
`0x32, 0x88, 0x81, 0x72`
- append a byte containing 0x01:
`0x32, 0x88, 0x81, 0x72, 0x01`
In this example, the resulting "Shockburst" address is: 0x3288817201.
**Additional example**, this time for inverter with serial number 99973104619:
The datasheet specifies the over-the-air packet format: "Most Significant Byte
(MSB) to the left" (cf [datasheet figure 11][3])
Address := Byte_4, Byte_3, Byte_2, Byte_1, Byte_0
("LSByte must be unique")
so 0x1946107301 results in
19 46 10 73 01 "on the wire"
Old-style NRF Libraries take uint64_t addresses. In this case, the correct
address to pass to the library would be (uint64_t)0x1946107301ULL.
The ["Optimized high speed nRF24L01+ driver"][4]
actually wants `uint8_t*`, which maybe makes more sense.
But apparently it still wants the bytes in order LSB to MSB (even though the chip will
then put them out in MSB-to-LSB order.
So in this case, the correct sequence of bytes to pass to the library
would be "\x01\x73\x10\x46\x19".
Figure 4 below is an annotated example of an "Enhanced Shockburst" packet as
seen on the air.
```
+----------+--------------------+--------------------+---------------------+------------+
| preamble | dst 5-byte-address | PCF (9-bit) | payload (>=1 bytes) | 2-byte-CRC |
+----------+--------------------+--------------------+---------------------+------------+
| | | e.g. 0x0d8: | | |
| 0x55 | addr[4]...addr[0] | 0b011011 00 0 | | |
| or | MSB ... LSB | len=27 PID nACK | | |
| 0xAA | | | | |
| | | e.g. 0x0da | | |
| | | 0b011011 01 0 | | |
| | | len027 PID nACK | | |
+----------+--------------------+--------------------+---------------------+------------+
PCF: Packet control field
PID: Packet IDentification (to detect/avoid duplicates), cycles through 0...3
FIGURE 4: Enhanced Shockburst On-Air Data Format
```
Messages
========
Initial protocol analysis focused on the data exchanged on link (C) in figure (3).
Not all the frames observed on this link will result in an actual RF transmission,
and some translation/mangling/processing happens inside the NRF24LE1E, in particular
- replacement of serial numbers
- recalculation of CRCs
These packets (which are all framed in 0x7e...0x7f bytes) are described in section
[Encapsulated Packets](#encapsulated-packets) below.
More recent efforts focus mainly on the actual "Enhanced Shockburst" packets
that are transmitted over the air. These packets are described in section
[Enhanced Shockburst Payloads](#Enhanced-Shockburst-Payloads), and the
information contained in this section is more up to date.
Encapsulated Packets
--------------------
These are packets as observed on Link (C) in figure (3).
Nachricht: DTU an WR: "Init" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
7E 07 00 00 00 00 00 00 00 00 00 07 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^
Bedeutung SOF MID WR ser# WR ser# ? CRC8 EOF
```
?
Nachricht: DTU an WR: "Init 2" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
7E 07 72 81 88 32 72 81 88 32 00 07 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^
Bedeutung SOF MID DTU ser# DTU ser# ? CRC8 EOF
Einheit BCD (letzte 8) BCD (letzte 8) ? ?
Beispiel 72818832 72818832 ?
```
Nachricht 0x80: DTU an WR: "Zeit setzen" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
|<-------------CRC16 'modbus' für CRC_M----------------->|
7E 15 72 22 02 00 72 22 02 00 80 0B 00 62 09 04 9b 00 00 00 00 00 00 00 00 F2 68 F0 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^^^^^^^^^^ ^^^^^ ^^ ^^
Bedeutung SOF MID WR ser# WR ser# CMD ? TIME (UTC) CRC_M CRC8 EOF
Einheit BCD (letzte 8) BCD (letzte 8) ? [s] HI LO
Beispiel 72220200 72220200 ? 2022-02-13
```
13:16:11
Nachricht 0x81: DTU an WR: "Anfrage DC-Daten" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
GD->NRF 7E 15 70 51 43 68 70 51 43 68 81 xx 7F ...... (NOCH NICHT VERIFIZIERT / GESEHEN)
^^^^^^^^^^^ ^^ ^^ ^^
| (wird von CMD CRC8 EOF
| NRF ersetzt) | (wird von NRF
v v neu berechnet)
on-air 15 70 51 43 68 70 53 54 53 81 BA
(payload) ^^^^^^^^^^^ ^^^^^^^^^^^
WR ser # DTU ser #
```
Nachricht 0x82: DTU an WR: "Anfrage AC-Daten" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
GD->NRF 7E 15 70 51 43 68 70 51 43 68 82 xx 7F ...... (NOCH NICHT VERIFIZIERT / GESEHEN)
^^^^^^^^^^^ ^^ ^^ ^^
| (wird von CMD CRC8 EOF
| NRF ersetzt) | (wird von NRF
v v neu berechnet)
on-air 15 70 51 43 68 70 53 54 53 82 B9
(payload) ^^^^^^^^^^^ ^^^^^^^^^^^
WR ser # DTU ser #
```
Nachricht 0x83: DTU an WR: "Anfrage DC-Daten" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
GD->NRF 7E 15 70 51 43 68 70 51 43 68 83 xx 7F ...... (NOCH NICHT VERIFIZIERT / GESEHEN)
^^^^^^^^^^^ ^^ ^^ ^^
| (wird von CMD CRC8 EOF
| NRF ersetzt) | (wird von NRF
v v neu berechnet)
on-air 15 70 51 43 68 70 53 54 53 83 B8
(payload) ^^^^^^^^^^^ ^^^^^^^^^^^
WR ser # DTU ser #
```
Nachricht 0x85: DTU an WR: "???" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
GD->NRF 7E 15 70 51 43 68 70 51 43 68 85 xx 7F ...... (NOCH NICHT VERIFIZIERT / GESEHEN)
^^^^^^^^^^^ ^^ ^^ ^^
| (wird von CMD CRC8 EOF
| NRF ersetzt) | (wird von NRF
v v neu berechnet)
on-air 15 70 51 43 68 70 53 54 53 85 BE
(payload) ^^^^^^^^^^^ ^^^^^^^^^^^
WR ser # DTU ser #
```
Nachricht 0xFF: DTU an WR: "???" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
GD->NRF 7E 15 70 51 43 68 70 51 43 68 FF xx 7F ...... (NOCH NICHT VERIFIZIERT / GESEHEN)
^^^^^^^^^^^ ^^ ^^ ^^
| (wird von CMD CRC8 EOF
| NRF ersetzt) | (wird von NRF
v v neu berechnet)
on-air 15 70 51 43 68 70 53 54 53 FF C4
(payload) ^^^^^^^^^^^ ^^^^^^^^^^^
WR ser # DTU ser #
```
Nachricht 0x01: WR an DTU: "Aktuelle DC Daten" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
7E 95 72 22 02 00 72 22 02 00 01 00 01 01 4c 03 bd 0c 46 00 b5 00 03 00 05 00 00 BD 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^ ^^
Bedeutung SOF MID WR ser# WR ser# CMD ? PV1.u PV1.i PV1.p PV2.u PV2.i PV2.p ? CRC8 EOF
Einheit BCD (letzte 8) BCD (letzte 8) ? [0.1V] [0.01A] [.1W] [0.1V] [0.01A] [.1W] ?
Beispiel 72220200 72220200 ? 33.2V 9.57A 317.2W 18.1V 0.03A 0.5W ?
```
Nachricht 0x02: WR an DTU: "Aktuelle AC Daten" (?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
7E 95 72 22 02 00 72 22 02 00 02 28 23 00 00 24 44 00 3C 00 00 09 0F 13 88 0B D5 83 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^^^^ ^^^^^ ^^^^^ ^^ ^^
Bedeutung SOF MID WR ser# WR ser# CMD ? ? ? AC.u AC.f AC.p CRC8 EOF
Einheit BCD (letzte 8) BCD (letzte 8) ? [0.1V] [0.01Hz] [0.1W]
Beispiel 72220200 72220200 ? 9284 60 231.9V 50.00Hz 302.9W
```
Nachricht 0x83: WR an DTU (?): "???" (nach CMD wäre das eher auch eine Antwort vom WR?)
```
----------------------------------------------------------------------------------------------------------------------------------------------
7E 95 72 22 02 00 72 22 02 00 83 00 03 00 83 03 E8 00 B2 00 0A FD 26 1E 7F
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^ ^^
Bedeutung SOF MID WR ser# WR ser# CMD ? ? ? ? ? ? CRC8 EOF
Einheit BCD (letzte 8) BCD (letzte 8) ?
Beispiel 72220200 72220200 ? 131 1000 178 10
```
Hinweise
--------
Die "on-air (payload)" Bytes geben nur die Nutzlast der gesendeten Shockburst-Pakete an.
Intern enthalten diese Pakete auch die Zieladresse, die Länge, eine CRC.
*****************************************************************************************************************************************************************************************
Enhanced Shockburst Payloads
----------------------------
- These are the packets that are exchanged between inverters and DTU via the Nordic
"Enhanced Shockburst" protocol.
- Each payload is preceded by a preamble, and terminated by a 16-bit CRC, as described
in the [Nordic datasheet][3]. See also figure 4 above.
```
CMD 0x80: DTU --> WR: "Set time/date" (?)
----------------------------------------------------------------------------------------------------------------------------------------------
|<-------------CRC16 'modbus' für CRC_M----------------->|
15 72220200 72220200 80 0B 00 62 09 04 9b 00 00 00 00 00 00 00 00 F2 68 F0
^^ ^^^^^^^^ ^^^^^^^^ ^^ ^^^^^ ^^^^^^^^^^^ ^^^^^ ^^^^^ ^^
Name MID DTU_SER# DTU_SER# CMD uk1 TIME (local) SEQ? CRC_M CRC8
Units see "addressing" ? [s-since-epoch] HI LO
Example 72220200 72220200 ? 2022-02-13
13:16:11
```
- This message will cause the inverter to transmit a CMD=0x01, CMD=0x02, and, occasionally, also a CMD=0x83 message
to the DTU with serial number DTU_SER#.
- Values of "0xb0, 0x00" and "0x11, 0x00" have been observed for "UK1". Their meaning is unknown.
- "SEQ" was observed to contain increasing numbers when sent by a Hoymiles DTU. In particular,
each issued "command" (e.g. "switch inverter on", "switch inverter off") appears to increase this
value. A constant value of 0x0000 or 0x0005 appears to work just fine.
- Repeatedly sending the same TIME information (instead of correctly increasing time)
[has been shown](https://www.mikrocontroller.net/topic/525778?page=2#7021386) to result
in identical behaviour, the inverter still replies as described above.
```
CMD 0x01: WR --> DTU: "Current DC data" (?) (shown for an HM-700)
----------------------------------------------------------------------------------------------------------------------------------------------
95 72 22 02 00 72 22 02 00 01 00 01 01 4c 03 bd 0c 46 00 b5 00 03 00 05 00 00 BD 7F
^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^^^^ ^^ ^^
NameMID WR ser# WR ser# CMD ? PV1.u PV1.i PV1.p PV2.u PV2.i PV2.p ? CRC8 EOF
Units BCD (letzte 8) BCD (letzte 8) ? [0.1V] [0.01A] [.1W] [0.1V] [0.01A] [.1W] ?
Example 72220200 72220200 ? 33.2V 9.57A 317.2W 18.1V 0.03A 0.5W ?
```
- The exact meaning of the contents of this message varies depending on inverter type. So far, the following variants have been observed:
- HM-400 (single channel):
- HM-700 (2-channel):
- HM-1500 (4-channel):
```
TODO TODO TODO
73109025 73109025 01 00 01 014F 0003 000B 0000 40AE 03AC 08E6 7C
^^^^ ^^^^ ^^^^ ^^^^ ^^^^
335 3 11 940 2278
33.5V 0.03A 1.1W 940W 22.78kW
95 71603546 71603546 01 00 01 015D 004D 00B3 010C 0270 0001 3419 64 B327 B327 1
^^^^ ^^^^ ^^^^ ^^^^ ^^^^
349 77 179 1 13337
34.9V 0.77A 1.79W 1 133.37kW
```
```
Nachricht 0x02: WR an DTU: "Aktuelle AC Daten" (?)
----------------------------------------------------------------------------------------------------------------------------------------------
7E 95 72 22 02 00 72 22 02 00 02 28 23 00 00 24 44 00 3C 00 00 09 0F 13 88 0B D5 83
^^ ^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^^^^ ^^^^^ ^^^^^ ^^
Bedeutung SOF MID WR ser# WR ser# CMD ? ? ? AC.u AC.f AC.p CRC8
Einheit BCD (letzte 8) BCD (letzte 8) ? [0.1V] [0.01Hz] [0.1W]
Beispiel 72220200 72220200 ? 9284 60 231.9V 50.00Hz 302.9W
```
- The exact meaning of the contents of this message varies depending on inverter type. So far, the following variants have been observed:
- ...
```
Nachricht 0x83: WR an DTU (?): "???" (nach CMD wäre das eher auch eine Antwort vom WR?)
----------------------------------------------------------------------------------------------------------------------------------------------
95 72 22 02 00 72 22 02 00 83 00 03 00 83 03 E8 00 B2 00 0A FD 26 1E
^^ ^^^^^^^^^^^ ^^^^^^^^^^^ ^^ ^^
Bedeutung MID WR ser# WR ser# CMD ? ? ? ? ? ? CRC8
Einheit BCD (letzte 8) BCD (letzte 8) ?
Beispiel 72220200 72220200 ? 131 1000 178 10
```
Legend
======
**MID**: Message-ID. Antworten haben Bit 7 gesetzt,
```
z.B. Frage 0x15 --> Antwort 0x95.
z.B. Frage 0x07 --> Antwort 0x87.
Für Kommunikation GD <--> NRF
```
**CMD**:
Befehl an den WR hat Bit 7 gesetzt
0x80 "Zeit setzen"
0x81 "Anfrage DC-Daten", erwartete Antwort: 0x01
0x82 "Anfrage AC-Daten", erwartete Antwort: 0x02
0x83 "?"
0x85 "?"
0xFF "?"
Antworten vom WR haben Bit 7 gelöscht:
0x01 "Aktuelle DC-Daten"
0x02 "Aktuelle AC-Daten"
**SOF**: Start-of-Frame 0x7e
**EOF**: End-of-Frame 0x7f
**CRC8**: CRC8 mit poly=1 init=0 xor=0, für alle Bytes zwischen SOF und CRC8.
Beispiel in Python:
```
>>> import crcmod
>>> f = crcmod.mkCrcFun(0x101, initCrc=0, xorOut=0)
>>> payload = bytes((0x95,0x72,0x22,0x02,0x00,0x72,0x22,0x02,0x00,0x83,0x00,0x03,0x00,0x83,0x03,0xE8,0x00,0xB2,0x00,0x0A,0xFD,0x26))
>>> hex(f(payload))
'0x1e'
```
**CRC_M**: CRC16 wie für "Modbus"-Protokoll, High-Byte gefolgt von Low-Byte
Beispiel in Python:
```
>>> import crcmod
>>> f = crcmod.predefined.mkPredefinedCrcFun('modbus')
>>> payload = bytes((0x0B,0x00,0x62,0x2F,0x45,0x96,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00))
>>> hex(f(payload))
'0x3bd6'
```
**TIME**: Aktuelle (DTU-)Zeit als Unix "time_t" (Sekunden seit 1970-01-01)
Glossary
========
**WR**: Wechselrichter (inverter)
**DTU**: Data Terminal Unit (?). Die Hoymiles-Bezeichnung für den Kommunikations-Master.
**BCD**: Binary Coded Decimal
Notizen
=======
0x014c = 332
0x03bd = 957
0x0c64 = 3172
0x6209049b = 1644758171
datetime.datetime.utcfromtimestamp(0x6209049b): datetime.datetime(2022, 2, 13, 13, 16, 11)
References
==========
- [1]: https://www.mikrocontroller.net/topic/525778 "The post that started the community effort"
- [2]: https://www.mikrocontroller.net "mikrocontroller.net"
- [3]: https://infocenter.nordicsemi.com/pdf/nRF24LE1_PS_v1.6.pdf "Nordic NRF24LE01+ datasheet"
- [4]: https://nrf24.github.io/RF24 "Optimized high speed nRF24L01+ driver documentation"
- [5]: https://github.com/grindylow/ahoy "AHOY Communications Project"
Revision History
================
2022-03-09 / Petersilie / erste Version
2022-03-10 / Petersilie / r2 / Nachrichten "02 28 23" und "82 00 03" ergänzt. Sauberer ausgerichtet. Python Beispiel für CRC.
2022-03-12 / Petersilie / r3 / Erste on-air Formate hinzu. CMD-IDs hinzu. Neue Nachrichten von arnaldo_g hinzu. Übersicht hinzu.
2022-03-15 / Petersilie / r4 / Nachricht 0x80: Mystery-Bytes am Ende "dechiffriert"
2022-03-16 / Petersilie / r5 / ESP ist ein ESP8266, nicht ESP32 (danke an @tbnobody)
2022-03-27 / Petersilie / all future revisions are now versioned via Git.