Older blog entries for DV (starting at number 233)

Wandering in embedded land: part 1, Midea 美的 aircon protocol

I have been a user of Arduino's for a few years now, I use them to
control my greenhouse (I grow orchids). This mean collecting data
for various parameters (temperature, hygrometry, light) and actionning
a collection of devices in reaction (fan, misting pump, fogging machine,
a heater). The control part is actually done by an NSLU2 which also
collects the data, export them as graph on the internet and allows me
to manually jump in and take action if needed even if I'm far away
using an ssh connection.

This setup has been working well for me for a few years but since our
move to China I have had an airon installed in the greenhouse like in
other parts of the home. And that's where I have a problem, this AC of
brand Midea (very common home appliance brand in China) can only be
controlled though a remote control. And until now that meant I had no
way to automtate heating or cooling, which is perfectly unreasonnable :-)

After some googling the most useful reference I found about those
is the Tom's
Site page on building a remote adapter
for those. It explained most
parts of the protocol but not all of them, basically he stopped at the
core of the interface but didn't went into details, for example didn't
explained the commands encoding. The 3 things I really need are:

  • Start cooling to a given temperature
  • Start heating to a given temperature
  • Stop the AC

I don't really need full fan speed control, low speed is quite sufficient
for the greenhouse.

Restarting the Arduino development

I hadn't touched the Arduino development environment for the last few
years, and I remember it being a bit painful to set up at the time. With
Fedora 17, things have changed, a simple

yum install arduino

and launching the arduino tool worked the first time, actually it asked me
the permission to tweak groups to allow me as the current user to talk
through the USB serial line to the Arduino. Once done, and logging in again
everything worked perfectly, congratulation to the packagers, well done !
The only sowtware annoyance is that is often take a dozen seconds between the
time an arduino is connnected or powered and when it appears in the
ttyUSB? serial ports options in the UI, but that's probably not arduino's
fault.

The arduino environment didn't really change in all those years,
the two notable exception is the very long list of different boards supoorted
now, and the fact that arduino code files are renamed from .pde to .ino !

Learning about the data emitted

The first thing needed was to double check the result from Tom with
our own hardware, then learn about the protocol to be able to construct
the commands above. To do this I hooked a IR receptor to the Arduino on
digital pin 3, the graphic below show the logic, it's very simple:

Then I loaded a modified (for IRpin 3) version of Walter Anderson's
IRanalyzer.pde
onto the Arduino and started firing the aircon remote control at the
receiver and looked at the result: total garbage ! Whatever the key
pressed the output had no structure and actually looked as random as
input without any key being pressed :-\

It took me a couple of hours of tweaking to find out that the metal
enclosure of the receiver had to be grounded too, the GRD pin wasn't
connected, and not doing so led to random result !

Once that fixed, the data read by the Arduino started to make some
sense and it was looking like the protocol was indeed the same as the
one described in Tom's site.

The key to the understanding of how the remote work is that it
encodes a digital input (3 Bytes for Midea AC protocol) as a set of
0 and 1 patterns each of them being defined by a 0 analogic duration
followed by a short anlogic pulse at 38KHz to encode 0, or a long
analogic pulse at 38KHz to encode 1:

Each T delay correspond to 21 pulses on a 38KHz signal, this is
then a variable lenght encoding

As I was making progresses on the recognition of the patterns sent
by the aircon I was modifying the program to give a more synthetic view
of the resulting received frames, you can use my own
IRanalyzer.ino
it is exended to allow recording of a variable number of transition,
detects the start transition as a 3-4 ms up, and the end as a 3-4 ms
down from the emitter, then show the transmitted data as bit field and
hexadecimal bytes:


Waiting...
Bit stream detected: 102 transitions
D U 1011 0010 0100 1101 1001 1111 0110 0000 1011 0000 0100 1111 dUD Bit stream end : B2 4D 9F 60 B0 4F !
4484 4324 608 1572 604 472 596 1580 600 1580 !
Waiting...

So basically what we find here:

  • the frame start markers 4T down, 4T up
  • 6 Bytes of payload, this is actually 3 bytes of data but after
    each byte is sent its complement is sent too
  • the end of the frame consist of 1T down, 4T up and then 4T down

there is a few interesting things to note about this encoding:

  • It is redundant allowing to detect errors or stray data coming from
    other 38KHz remotes (which are really common !)
  • All frames are actually sent a second time just after the first one,
    so the amount of redundancy is around 4 to 1 in the end !
  • by reemitting inverted values, the anmount of 0 and 1 sent is the
    same, as a result a frame always have a constant duration even if the
    encoding uses variable lenght
  • A double frame duration is around : 2 * (8 + 8 + 3*2*8 + 3*4*8 + 1 + 8) * 21 / 38000 ~= 186 ms

The protocol decoding

Once the frame is being decoded properly, we are down to analyzing
only 3 bytes of input per command. So I started pressing the buttons
in various ways and record the emitted sequences:


Cool 24 fan level 3
1011 0010 0100 1101 0011 1111 1100 0000 0100 0000 1011 1111 B2 4D 3F C0 40 BF
Cool 24 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 0100 0000 1011 1111 B2 4D 9F 60 40 BF
Cool 20 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 0010 0000 1101 1111 B2 4D 9F 60 20 DF
Cool 19 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 0011 0000 1100 1111 B2 4D 9F 60 30 CF
Heat 18 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 0001 1100 1110 0011 B2 4D 9F 60 1C E3
Heat 17 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 0000 1100 1111 0011 B2 4D 9F 60 0C F3
Heat 29 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 1010 1100 0101 0011 B2 4D 9F 60 AC 53
Heat 30 fan level 1
1011 0010 0100 1101 1001 1111 0110 0000 1011 1100 0100 0011 B2 4D 9F 60 BC 43
Stop Heat 30 fan level 1
1011 0010 0100 1101 0111 1011 1000 0100 1110 0000 0001 1111 B2 4D 7B 84 E0 1F
Cool 28 fan 1
1011 0010 0100 1101 1001 1111 0110 0000 1000 0000 0111 1111 B2 4D 9F 60 80 7F
Stop Cool 28 fan 1
1011 0010 0100 1101 0111 1011 1000 0100 1110 0000 0001 1111 B2 4D 7B 84 E0 1F

The immediately obvious information is that the first byte is the constant
0xB2 as noted by Tom's Site. Another thing one can guess is that the command
drom the control is (in general) absolute, not relative to the current state
of the AC, so commands are idempotent,if it failed to catch one key, it will
get a correct state if this is repeated, this just makes sense from an UI
point of view ! After a bit of analysis and further testing the
code for the 3 bytes seems to be:

[1011 0010] [ffff 1111] [ttttcccc]

Where tttt == temperature in Celcius encoded as following:

17: 0000, 18: 0001, 19: 0011, 20: 0010, 21: 0110,
22: 0111, 23: 0101, 24: 0100, 25: 1100, 26: 1101,
27: 1001, 28: 1000, 29: 1010, 30: 1011, off: 1110

I fail to see any logic in the encoding there, I dunno what the Midea
guys were thinking when picking those values. What sucks is that the protocol
seems to have a hardcoded range 17-30, while basically for the orchids
I try to keep in the range 15-35, i.e. I will have to play with the
sensors output to do the detection. Moreover my test is that even when
asked to keep warm at 17, the AC will continue to heat until well above
19C, I can't trust it to be accurate, best is to keep the control and logic
on our side !

cccc == command, 0000 to cool, 1100 to heat, 1000 for automatic selection
and 1101 for the mode to remove moisture

Lastly ffff seems to be the fan control, 1001 for low speed, 0101 for
medium speed, 0011 for high speed, 1011 automatic, and 1110 for off. There
is also a mode which is about minimizing energy, useful at night, where
the fan is quite slower than even the low speed, but i didn't yet understood
how that actually work.

There is still 4 bytes left undeciphered, they could be related to 2
function that I don't use: a timer and oscilation of the air flow, I
didn't try to dig, especially with a remote control and documentation
in Chinese !

Last but not least: the stop command is 0xB2 0x7B 0xE0, it's the same
whatever the current state might be.

At this point I was relatively confident I would be able to control the
AC from an Arduino, using a relatively simple IR LED control, it ought to
be a "simple matter of programming", right ?

Well, that will be the topic of the next part ;-) !

This entry will be kept at http://veillard.com/embedded/midea.html.

28 May 2012 (updated 28 May 2012 at 14:51 UTC) »

libxml2-2.8.0:

Finally ! Last release was 18 months ago, and after a good week of testing the new libxml2 is out. I tried to process all pending patches fom GNOME bugzilla, most of them went in, after more or less massaging :) . There is also a new feature, transparent support for lzma compression, which is quite efficient in terms of squeezing the input without taking too much cycles compressing (and decompressing). Still this is mostly a bugfix release, but with enough changes that it is worth bumping the middle version. Get it from xmlsoft.org.

libvirt 0.9.12

Libvirt flow of releases continues, the current version is 0.9.12 and I try to keep a release cycle close to one per month. I extracted the following graph of commits activity using gitstats:

The project had a peak of activity one year ago and this is slowly decreasing now, maybe the poject finally reached the main goals that contributors were looking for and while there is still a number of things left to do we are finally entering the stabilization phase of the project. Hard to tell, but maybe I will start to decrease the rate of the release, but that need discussion on the list first :-)

LinuxCon Japan

I have been invited t talk again at LinuxCon Japan next week. Last year was very interesting, with good discussions in many areas, and I hope this will be the case again this year. It is also good to meet with some users face-to-face as oral communication is so different from the way we proceed when using email. For me this is also a way to get different perspectives and look at the project and the whole ecosystem to try to get a more complete picture.

Fedora 17

I usually update my systems only every other Fedora release but I think this time I'm gonna upgrade earlier to play with OpenVSwitch which sounds like a good way to solve a number of annoyances with networking setups :-)

Whoops: water resilient Linux

I forgot my motorola android phone in my pocket while sitting in the hot spring pool yesterday, I quickly realized
though, rushed it out of the water, disassembled, removed the battery in a hurry. Funilly the display did stay on as "normal" but the system didn't like this. Dried it and kept the pieces overnight on the UPS to dry out thoroughly, and replugged SIM and battery this morning ... working :-) . Except for that incident the hot spring trip was great, and Cheng definitely loves playing in the water !

libxml2

In the category of stable software, libxml2 is relatively quiet though I need to do a release incorporating various XPath bugs. I'm quite slow at the maintenance those days (which I think is fine considering the project is really in maintenance mode), but I should get that release out ! Someone kick me :-)

libvirt

In contrast libvirt development is *not* slowing down as I was expected. We still roll a release a month, each incorporating 300+ patches, and the trend is that the patch number is still increasing, leading to more volume on-list. I was naively thinking that since most APIs were done, the volume would start to decrease around one year ago, but nope, people still want more features, better hypervisors support, etc... and that's just fine :-)


libstoragemngt

One of the new library project I am looking at now is libStorageMgmt the goal is to provide management APIs for those external storages arrays like SANs. Right know they each have their own specific ways to manage partitionning, provisionning, etc... and we hope to be able to improve that, but it will take a while. The project is starting and currently looking at design options, if that field interrest you, get on the list and provide some feedback, it may be more useful to you when it gets usable :-)


15 Apr 2010 (updated 15 Apr 2010 at 09:47 UTC) »

Ouch, it seems I didn't wrote any blog entry for nearly 2 years, since our wedding. Well I would say that I have been fairly busy :-) !

Baby Cheng 程

Our son is born end of last august, so obviously this tends to keep the parents busy, but it's such a pleasure too ! picture of cheng

libvirt project

this has been a wild ride ! The project has grown a lot, in community and contributor size, as well as code and features during the last couple of years. It acquired support for most well known virtualization hypervisors (QEmu/KVM, Xen, ESX, VitualBox, ...) the only notable exception being Microsoft Hyper-V I think. Our latest release is 0.8.0 so we are not at 1.0 yet, but we are really getting closer, for example this release now adds snapshotting support which was a big feature request.

About updates to Fedora

There was a discussion last month about forbidding direct built to the stable branches, and I really think this makes sense, except for security erratas, one really should not push directly an updated build onto the users without some testing. But on the other hand we entered beta on Fedora 13 and apparently it's too late to push libvirt latest release and features, meaning users who don't try to update either via the virt-preview special repository or by rebuilding from the source rpm, will see libvirt features when Fedora 14 is being released i.e around the end of the year. I see people bouncing between both extreme of the features vs. stability tradeoff, and I wonder if there isn't a better way than building absolute rules. Maybe a general profile per machine where the user indicates whether he want updates, just bug fixes or just security errata, and let him do the picking. This would require some policy from package builders, and some changes at the repo level I guess, but having the user ultimately in control might be worth it.

13 Jun 2008 (updated 13 Jun 2008 at 13:44 UTC) »

A new release: libvirt-0.4.3

After two months a lot of patches had accumulated, including a lot of improvements for Xenner and Linux container support. But this release brings a massive set of code cleanup, and just looking at the patches there there is a lot of obscure case failures which should now be properly handled (or at least better handled, like out of memory situations). I'm pushing testing updates for F-8 and F-9 if you have time and use virtualisation please review them, thanks !

A new name ?

I got married to Miss Wei LI last saturday, it was kept a simple family event with just a few close friends, everything went well except for a bit of rain ! For the name I wouldn't mind being called Mr Li, but it's probably not very practical at this point (ah and good luck getting the li.com domain, and I guess hijacking li.org would not be well accepted either ;-)

A very simple picture, people interested for more should know where to look for already. Oh it also made me fight with the Panasonic HDC-SD9 'new' MPEG-4 output, to get videos to render properly on that other OS, I didn't expect to learn so much about video format so quickly. I will post recipe and scripts later.

small
wedding picture

Releases

I pushed a bunch of releases on Tuesday, trying to catch the Fedora-9 train (I nearly missed it, it led to a not so fun curl_is_failing_to_upload debug session which led to nss3 for firefox3 is not compatible with nss3 for fedora8 curl), thanks to everybody who helped catch that train !

The releases are mostly bugfixes, libvirt-0.4.2 leading the pack, but libxml2-2.6.23 has a lot of fixes too thanks to various people reporting bug and giving patches, notably the Huawei team. Libxslt-1.1.23 includes the dozen or so fixes since last summer.

Developments

Clearly libxml2 and libxslt are in maintance mode, the focus is on libvirt, maybe I will just add support for the latest Proposed Recommendation of XML-1.0 in libxml2 before the Summer.

For libvirt, clearly we need to extend the number of hypervisor supported, maybe update and clean up the OpenVZ support too. IBM is actively contributing the Linux Container driver, I just commited a second set of patches today, you can expect good support in Fedora 10 I guess. On the high end side Sun just posted the patches for the lDOM virtualization on their Niagara based machines, lot of patch reviews those days. I also want to get a complete set of bindings for Java integrated, and now that Fedora java packaging guidelines are out, this is a good opportunity to add this.

History meme

that one is interesting, here is my contribution, as you can see I'm an old fashionned old fart, main workstation at home:

paphio:~ -> history | awk '{a[$2]++ } END{for(i in
a){print a[i] " " i}}'|sort -rn|head
319 vi
257 ssh
255 cd
156 cvs
130 make
125 ls
79 svn
60 scp
48 su
43 ping
paphio:~ ->

and on my second workstation in Annemasse:

wei:~ -> history | awk '{a[$2]++ } END{for(i in
a){print a[i] " " i}}'|sort -rn|head
362 vi
263 cd
262 make
136 cvs
115 svn
105 ssh
78 ls
67 scp
40 xmllint
38 grep
wei:~ ->

Happy new Year

yeah it's either really late or a bit too early for Chinese New year, proof that I should blog more often !

backlog

I have been away for the last month and a half, first vacation in China, and then in Paris for Solution Linux expo. As a result an awful lot of mail piled up in various folders that I didn't really had time to process, I am slowly trying to go though them, so if you get replies from a mail in November that's normal, or if you didn't get a reply, it's probably a good idea to ask again !

libvirt and Co.

libvirt development is progressing fast thanks to DanB, Rich, Jim and the many people porting, testing and using it. I really hope we will soon have easy support for most platform - including Windows and OS-X - at least as a client to remotely access the virtualization servers, Rich did most of the work. The development of the CIM provider seems to be progressing nicely thanks to DanS and the IBM'ers :-). During last week Solution Linux i got asked a few times about P2V tools i.e. tools allowing to automatically save the state of a server to allow to start it as a virtualized domain, Rich just released virt-p2v 0.9.1, it is still experimental, what it needs now is a wider range of testing to find and remove limitations, give it a try and provide feedback, thanks !

For libvirt core we still have a number of pending extensions, like support for new hypervisors, adding APIs for storage management which have been worked on by DanB, we also need to finalize a good XML scheme for 'partition' based virtualization, probably update OpenVZ, and add sound and more USB support. I will post on the list soon to try to clarify the roadmap, much work ahead !

Solution Linux

SL 08 was last week, it's the Linux event of the year where I'm sure to go, it was good seeing people around the Fedora and Gnome booth, and of course Red Hat one. My presentation on virtualization went well, even if it was more than 2 hour long, the PDF is available and a lot of people seems to have fetched it already, I assume it's a positive feedback from the audience :-)

China

I was for a month in China, what a blast! It's impossible to try to describe in just a few lines, but we went from the very cold Dalian and Beijing to tropical Xishuanbanna, most of the time within the chinese family and environment, picking only a few selected tourist spot. Things like Shanghai museum, the Great Wall or the Forbidden city are definitely must-go, but to me the most amazing was to stay and live with the chinese inside the family. The cultural gap is huge, there is so much to learn, too bad mandarin is so hard to an occidental brain (and ears), very challenging, but also very fun and enjoyable !

coral bleach

ncm, I think this may be an error of interpretation, the coral turns white when it dies, I mean when the organism dies. They form a symbiosis with alguaes and those give the color, white coral appears when the animal dies, you just have the skeletton left. White coral (with a few exceptions which are naturally white) is just like the bones left from a dead animal, it may result from increased acidity of water, but also from any other deadly condition. For example when overheating, if water temp reaches around 42 degrees, the coral rejects the alguae then dies, it can be quite dramatic because suddenly very large areas whithen suddenly, it's all dead and will take maybe half a century to get back in proper shape :-(

libvirt progresses

libvirt development progresses rapidly, as the traffic on the list can testify, we now have native secure remote operations, and there is ongoing work to add OpenVZ support. I hope to increase coverage of more virtualization engine, so far it looks good, we don't seems to have serious API design problems.

gamin

I made a new release 0.1.9 last week with some bug fixes and portability improvements.

19 Jun 2007 (updated 19 Jun 2007 at 22:36 UTC) »

One year in review

I didn't blogged for a year precisely. No I didn't hide in a hole or something but I just didn't really feel the need. I still did a few things:

  • the libvirt project is my main work item, 15 releases in the last year, now supports QEmu and KVM, many contributors and well people are using it, so that's good.
  • the libxml2 and libxslt libraries have seen their development slowing down, partly because they are mature enough, partly because I have less time, and Kasimier one of the main contributor lost his job and left the project. But Bill is still around and helps a lot !
  • travelling, I had the opportunity to travel for work and also for vacations, went to Singapore twice, also Malaysia, had some fantastic time in Borneo, visited Bangkok, but didn't like Manilla too much. Also spent some vacations in Costa Rica, and went to USA and Canada for work. I really enjoy going to Asia and meet people there.
  • I am getting crazy about orchids, my collection is growing fast, maybe a bit too fast, but I'm rewarded by fantastic flowers, and I'm trying more and more to travel to places where I can see them growing in their natural habitat, unfortunately they are seriously endangered, and it can be fairly hard to still find them in the wild.
  • I also went to badminton training again this year but competition is really getting a bit too hard, and my body disagrees at time, I think I will slow this down a bit next year.

Overall quite a bit happened but no revolution, the most scary thing is that time flies faster and faster, so no promise about increase of blogging frequency !

On broken XML processing with libxml2

The XML spec is very clear about this:

Once a fatal error is detected, however, the processor MUST NOT continue normal processing (i.e., it MUST NOT continue to pass character data and information about the document's logical structure to the application in the normal way).

That's the default behaviour of libxml2, but for the purpose of helping recover occasionally corrupted data, I provided an XML_PARSE_RECOVER parser flag allowing to correct some of the trivial errors, for example if using xmllint --recover. Now the problem is that people start using this flag for default processing, and that's something I said many time I didn't want to happen. If blog generator can't output correct well formed XML, then they need to be fixed (and that's true for all other XML generators). The intent of the drastic rule in the XML spec is cristal clear, to avoid data corruption and data loss by forcing non conformant behaviour to be fixed at the source, not at the consumer level.

I am gonna give a hard time to those who abused this feature of libxml2, because while not completely disabling the option, I will make sure it's not used repeatedly, I'm still looking at the best way. Feedback welcome on this issue, preferably on the mailing-list.

224 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!