5" Commands

[MrFr1day]

Hi,

Following @gerph suggestion, I managed to get, and test most commands (as far as I know, only "add-file" is missing), I can't implement this in code for the library, but I can post what I know, and test if you implement it.

Based on my tests, I think the way their app (UsbMonitorL) works is something like this:

1- Gather the selected PC info.
2- Render a full image in BBGGRRAA format.
2a- Play the Video (if one was set as a background).
3- Send the rendered image using a "Full_Image" command.

Then loop the following:
4- Gather PC info again.
5- Render another full image.
6- Compare it with the current image, then generate the changed-pixels message.
7- Send the changed-pixels message using an "Update_Image" command.
8- Apply the changes to the image stored in the app so it can be synced with the monitor.

---

General Commands:

• Get the device name/type and firmware (and maybe initialize):
  01 ef 69 00 00 00 01 00 00 00 c5 d3

• Set brightness, L=1-255:
  7b ef 69 00 00 00 01 00 00 00 (L)

• Set options, M=StartMode(1=Default, 2=PlayImage, 3=PlayVideo), F=Album-Flip 180 (1=Enable), S=SleepAfter in minutes (0=Disabled, app range 1-10):
  7d ef 69 00 00 00 05 00 00 00 2d (M) 00 (F) (S)

• TurnOff monitor (it will change it's port number):
  83 ef 69 00 00 00 01

• Restart monitor:
  84 ef 69 00 00 00 01

---

Storage Commands:

• Query Internal/SDCard storage usage, the reply is (InternalTotal-InternalUsed-InternalFree-SDTotal-SDUsed-SDFree):
  64 ef 69 00 00 00 01

• List Images/Videos in Internal Storage (the folder structure is already created), the reply is a list of files (result:dir:file:f1.ext/f2.ext/fN+.ext/):
  65 ef 69 00 00 00 0a 00 00 00 2f 72 6f 6f 74 2f 69 6d 67 2f
65 ef 69 00 00 00 0c 00 00 00 2f 72 6f 6f 74 2f 76 69 64 65 6f 2f

• List Images/Videos in SD Card:
  65 ef 69 00 00 00 10 00 00 00 2f 6d 6e 74 2f 53 44 43 41 52 44 2f 69 6d 67 2f
65 ef 69 00 00 00 12 00 00 00 2f 6d 6e 74 2f 53 44 43 41 52 44 2f 76 69 64 65 6f 2f

• Get a video file size P=Path, the reply is the size in bytes:
  6e ef 69 00 00 00 13 00 00 00 (P)

• Delete a video file, P=Path:
  66 ef 69 00 00 00 14 00 00 00 (P)

---

Media Commands:

• Play Video, P=Path, the reply is (play_video_success):
  78 ef 69 00 00 00 13 00 00 00 (P)

• Stop Video:
  79 ef 69 00 00 00 01

• Display a full image (for an image background):
  c8 ef 69 00 17 70

• Display an initial full image overlay (for a video background):
  ca ef 69 00 17 70

• Display a changed-pixels image (for an image background), S=Image-message bytes count (2bytes), N=Serial number of the image starting from 0 (4bytes):
  cc ef 69 00 00 (S) 00 00 00 (N)

• Display a changed-pixels image (for a video background), S=Image-message bytes count (2bytes), N=Serial number of the update starting from 0 (4bytes), V=visible-pixels info bytes count (4bytes):
  cc ef 69 00 00 (S) 00 00 00 (N) 00 00 (V)

• Query monitor render status, reply is (needReSend:0|renderCnt:X), "needReSend" will be 1 if a changed-pixels message was missed, and "renderCnt" will increase with each frame rendered by the monitor it self:
  cf ef 69 00 00 00 01

---

Image Messages (for an image background):

• Full-Image message:
  Basically, list all the pixels in BBGGRRAA hex format (e.g blue: FF0000FF)

• Changed-pixels message:
  S(6bytes) C(4bytes) P(6bytes per pixel), where:

S=starting pixel position
C=Count, how many pixels
P=PixelColors in BBGGRR format

There is no X and Y, so the (S) value must be calculated (S = ((X * 800) + Y), e.g. to display 3 White pixels starting at X=20, Y=30:
003E9E 0003 ffffff ffffff ffffff

If it's only a single pixel, then the count variable (C) is omitted, and the position variable's (S) first hex digit is set to "8", e.g. to display a single White pixel at X=20, Y=30:
803E9E ffffff

---

Image Messages (for a video background):

• Full-Image message:
  Same as the Image-background, but an extra command is needed to define the visible pixels, V=visible-pixels-info bytes count (4bytes):
  d0 ef 69 00 00 (V)

As for the visible-pixels-info itself, it's the same as the changed-pixels message, but without the PixelColors variable (P), then it needs to end with "00ef69", e.g. to make only 3 pixels visible starting at X=20, Y=30:
003E9E 0003 00ef69

• Changed-pixels message:
  It differ from the Image-background version in two ways:
  1- Just like the Full-Image, the visible pixels needs to be appended to the message
  2- The Alpha channel is somehow added to the pixel colors (BBGGRR) value, but I don't know how exactly, my guess from my tests is that the Alpha channel hex digits replaces the second hex digit of Blue and Green channels.

Note that any pixel not defined in the "visible-pixels-info" will not be shown, regardless of its transparency, and any pixel added at any point can be set to visible/hidden at any changed-pixels message.
In other words, not marking a pixel as visible, does not delete it, just hides it until set to visible again by a changed-pixels message.

---

Notes:

• The app always starts with the "Get_Device" command, might be used to determine the supported commands, or it could also functions as an initialize command.
• There are commands that the app sends but don't seem to do anything (I can skip them with no visible problems).
• All messages must be 250 bytes of size, or multiple of it.
• All commands must be 250 bytes by themselves, so they must be padded without counting other messages within them.
• Every 250th byte is unused, so an extra byte must added.
• Every reply from the monitor (except for two commands, "Get_Device" and "query storage usage") is padded to 1024 bytes.
• The amount of transparent pixels will affect the frame-rate of the video background.
• Some commands can be truncated and still work, but I opted to list them as the app sent them.

[MrFr1day]

Here is a working example (for an image background, and based on how their app communicate with the monitor):

from math import ceil
import serial
import cv2
import atexit

class CMD():
    Get_Device = '01ef6900000001000000c5d3'
    Update_IMG = 'ccef690000'
    Stop_Video = '79ef6900000001'
    Display_Full_IMAGE = 'c8ef69001770'
    Query_Render_Status = 'cfef6900000001'

class Unknown():
    PreImgCMD = '2c'
    Media_Stop = '96ef6900000001'
    PostImgCMD = '86ef6900000001'
    OnExit = '87ef6900000001'
  
def OnExit():
    SendMSG(Unknown.OnExit)
    lcd_serial.close()


def ReadReply():
    response = lcd_serial.read(1024).decode('utf-8')
    print(response)

def SendMSG(MSG, PadValue = '00'):
    if type(MSG) is str: MSG = bytearray.fromhex(MSG)

    MsgSize = len(MSG)
    if not (MsgSize / 250).is_integer(): MSG += bytes.fromhex(PadValue) * ((250 * ceil(MsgSize / 250)) - MsgSize)
    
    lcd_serial.write(MSG)
    return
    
    #I didn't notice any speed difference in splitting the messages, but their app had random splits in their messages ...
    MsgLimit = 111000
    MSG = [MSG[i:i + MsgLimit] for i in range(0, len(MSG), MsgLimit)]    
    for part in MSG: lcd_serial.write(part)


def GenerateFullImage(Path):
    image = cv2.imread(Path, cv2.IMREAD_UNCHANGED)        
    if image.shape[2] < 4: image = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)

    image = bytearray(image)
    image = b'\x00'.join(image[i:i + 249] for i in range(0, len(image), 249))
    return image

def GenerateUpdateImage(Path, x, y):
    image = cv2.imread(Path, cv2.IMREAD_UNCHANGED)
    width = image.shape[0]
    height = image.shape[1]

    MSG = ''
    for h in range(height):
        MSG += f'{((x + h) * 800) + y:06x}'  + f'{width:04x}'       
        for w in range(width): MSG += f'{image[h][w][0]:02x}' + f'{image[h][w][1]:02x}' + f'{image[h][w][2]:02x}'

    UPD_Size = f'{int((len(MSG) / 2) + 2):04x}' #The +2 is for the "ef69" that will be added later
    
    if len(MSG) > 500: MSG = '00'.join(MSG[i:i + 498] for i in range(0, len(MSG), 498))
    MSG += 'ef69'

    return MSG, UPD_Size


lcd_serial = serial.Serial("COM3", 115200, timeout=1, rtscts=1)
atexit.register(OnExit)

SendMSG(CMD.Get_Device)                     #Skippable 
ReadReply()
SendMSG(CMD.Stop_Video)                     #Skippable if there is no video playing now
SendMSG(Unknown.Media_Stop)                 #Skippable, might be for album playback
ReadReply()                                 #The reply should be "media_stop"
SendMSG(Unknown.PreImgCMD, '2c')            #Skippable, the app pads it using "2c" instead of 00

SendMSG(CMD.Display_Full_IMAGE)
image = GenerateFullImage(r'F:\1.png')
SendMSG(image)
ReadReply()                                 #The reply should be "full_png_sucess"
SendMSG(Unknown.PostImgCMD)                 #Skippable 

SendMSG(CMD.Query_Render_Status) 
ReadReply()                                 #The reply should containts (needReSend:0) to confirm all message are read/deliverd in order

MSG, UPD_Size = GenerateUpdateImage(r'F:\2.png', 30, 50)
SendMSG(CMD.Update_IMG + UPD_Size)
SendMSG(MSG)

SendMSG(CMD.Query_Render_Status) 
ReadReply()                                 #The reply should containts (needReSend:0) to confirm all message are read/deliverd in order

As you have guessed by now, I'm not a programmer by trade, and this is my first Python code, and working with hex, images, and serial-comms in this way, so I want to ask, did I waste my time decoding a known-standard for their changed-pixels messages, or did they actually make their own?

Also, using the code I posted, it takes about 0.6 seconds to convert and send a full image (800x480), is this monitor faster than the 3.5 model?

[gerph]

That's an impressive amount of decoding for the device, and it seems a little different to the interface that we've seen so far. I've got a device on order so fingers crossed I'll be able to play with that 'soon' (for whatever value of 'soon' AliExpress can give me!).

Well done in getting that far - it looks very useful to produce an update to the libraries!

[arthurferrai]

I have one of these, and I think I can help reverse-engineering this display... Which data sniffer are you using?

[MrFr1day]

Thanks for the offer, but as I said in the OP, aside from adding media files to it, I think I've already decoded all the commands, and I've been using it for some time now.

[sambasan]

Thank you very much for the reverse engineering work so far.
I'm currently writing 5" display support for this software. Still at prototype level and can take some time to finish but anyways. The plan is to also support video backgrounds in themes.
I've also got the display and can help reverse engineer the commands a bit further if needed.
I'm using a generic serial analyzer software that can tap into the communication that is happening on USB over the CDC driver.

Findings so far:
All commands requiring path, for example:

• Get a video file size P=Path, the reply is the size in bytes:

6e ef 69 00 00 00 13 00 00 00 (P)

Actually have the path length in characters, meaning that the correct decoding would be:
6e ef 69 00 00 00 (L) 00 00 00 (P)
L: Path length in characters for example for /mnt/SDCARD/video/earth.mp4 this would be 1b (dec: 27)
P: Path itself (ascii encoded to hex)

Additionally I decoded the video upload process and got a simple upload script working. I'll post the full code example a bit later but in general it goes like this:

1. Check if file exists and its size with get video file size
2. Send upload video command (see below), response should be "create_success"
3. Send raw video bytes (in 250 byte chunks / packets, see sendMSG by MrFr1day), response should be "file_rev_done.img_show"
4. Check file size after upload
5. Send init commands again
6. Send play video, response should be "play_video_success"

Upload video command:

6f ef 69 00 00 00 (L) 00 00 00 (P) (V)
L: Path length in characters for example for /mnt/SDCARD/video/earth.mp4 this would be 1b (dec: 27)
P: Path itself (ascii encoded to hex)
V: Video bytes count with endianness flipped so video with 1069828 bytes (105304 hex) would be 045310 hex you can do this with: struct.pack('<i', video_size_bytes).hex()

Code I used to split the video file to chunks:

def read_in_chunks(file_object, chunk_size=249):
    while True:
        data = file_object.read(chunk_size)
        if not data:
            break
        yield data

def upload_video(local_path):
    with open(local_path, "rb") as video_file:
        for packet in read_in_chunks(video_file):
            send_msg(packet)

---

Notes:

I expect these to work pretty similarly for internal / SD card storage with only path being the differing part.

[alexwbaule]

Can i help with this ? I buy it one with 5 pol screen.

I made a usbcap in windows to get commands, etc....

[alexwbaule]

I want to stick with python , im using linux, and i want to make this work on it.

There is some reverse engineering processing or some documentation about the 5 pol device ?

Using the code that @MrFr1day write, change port, its worked. But, there is more commands to get ?

[arthurferrai]

That's what we have for now... The original executable is obfuscated, so it's a mess to read.

[alexwbaule]

Im doing some reverse engineering with the original executable, and wireshark capturing usb packs.

When i finish i will post here.

I found some news about play videos, and another things.

[ErikApption]

@alexwbaule - what is missing from the code from @MrFr1day? I also tried reverse engineering the original code but it has been obfuscated so it was not very helpful imo

[alexwbaule]

Ok ! Lets share what i found until now....

import time
import serial

def ReadReply():
    retries = 0
    while lcd_serial.in_waiting == 0:
        print("waiting to read...")
        retries += 1
        time.sleep(0.1)
        if retries == 50:
            print("I Can't read.. wait 50 times...")
            break
    response = lcd_serial.read_all().decode('utf-8')
    print("Received: [{}]".format(response))

def SendFromFile(m):
    print("Sended: {}".format(m))
    msg = bytearray.fromhex(m)
    print("Size Msg Sended {} and Waiting... {} - {} ({})".format(len(msg), lcd_serial.out_waiting, lcd_serial.in_waiting, lcd_serial.is_open))
    try:
        s = lcd_serial.write(msg)
        lcd_serial.flushOutput()
        print("Size Msg Sended ({}) ({})".format(s,lcd_serial.out_waiting))
        while lcd_serial.out_waiting > 0:
            print("waiting to finish write...")
            time.sleep(0.1)
    except serial.serialutil.SerialException as err:
        print("Error...Timeout {}".format(err))

lcd_serial = serial.Serial("/dev/ttyACM0", 115200, timeout=1, rtscts=1)

f = open('SendReceiveComplete.txt', mode="r")
contents = f.readline()
while contents:
    cmd, line = contents.split("\t")
    if cmd == "0x00":
        SendFromFile(line.rstrip())
        #time.sleep(0.2)
    if cmd == "0x01":
        ReadReply()
    contents = f.readline()
print("Finished...")
f.close()
lcd_serial.close()

With this code (sorry !! im not a python coder or developer) and this file in attachment, you can "play a update image stream".

(stream file)

How i get this file ?

Using wireshark + some commands to export.

First, set this filter on wireshark (and use one port with only one item attached).

(usb.transfer_type == URB_BULK and usb.irp_info.direction == 0 and usb.endpoint_address.direction == 0) or (usb.transfer_type == URB_BULK and usb.irp_info.direction == 1 and usb.endpoint_address.direction == 1)

Second, with the "action" that you want to capture done, you can save this capture. (stop the capture before, and go to File -> Export Specified Packages).

Once you saved the capture, you can run this command to save only the streaming:

"C:\Program Files\Wireshark\tshark.exe" -T fields -e usb.irp_info.direction -e usb.capdata -r All.pcapng

The usb.irp_info.direction inform the direction (0x00 from your host | 0x01 from your screen device), and the usb.capdata is the "stream" itself.

What i found ? just one command that is 7b ef 69 (set the brightness) before the PostImgCMD.

I tried to use the GenerateUpdateImage in a loop, but at the second time that i call, the "needReSend" is allways 1, and i cant update the image (i build 3 equals images, with the number 1, 2 and 3), but using the captured stream, the update works ok, so, there is something missing that i can't say what is LOL....

If someone can help to reverse this stream to some "image file" (maybe doing the reverse of GenerateUpdateImage does)...

[alexwbaule]

Hi...

Using this
https://github.com/icsharpcode/ILSpy/releases/tag/v8.0

@arthurferrai and @ErikApption
I get some decompiled itens , show the draw itens, etc, some strings to send command.
To get all strings, its easy, the function is in the code.

internal static string b(string A_0, int A_1)
{
	char[] array = A_0.ToCharArray();
	int num = (int)((nint)(545885363 + A_1) + (nint)56 + 3);
	int num3 = 0;
	if (num3 >= 1)
	{
		goto IL_0022;
	}
	goto IL_0055;
	IL_0055:
	if (num3 >= array.Length)
	{
		return string.Intern(new string(array));
	}
	goto IL_0022;
	IL_0022:
	int num2 = num3;
	char num4 = array[num2];
	byte b = (byte)((num4 & 0xFFu) ^ (uint)num++);
	byte b2 = (byte)(((int)num4 >> 8) ^ num++);
	byte num5 = b2;
	b2 = b;
	b = num5;
	array[num2] = (char)((b2 << 8) | b);
	num3++;
	goto IL_0055;
}

All the strings that has the global::쏔.b("<OBFUSCATED STRING>", a_); the a_ is in the start of the function, like here:

I Dont have .NET installed in my Windows, so i cant rebuild the strings... but i think its a start point.

[sambasan]

Hi, I just wanted to clarify that it is possible to send only the changed pixels over and over again if you set the index (serial number V in MrFr1day notes) properly in the changed pixels update command itself. It is possible to use a running number (bigger than the last one sent)

It's what I'm currently doing in the prototype implementation.

I'm currently close to getting the driver to do changed pixels over video background.

Also the currenty used serial device locking does seem to work properly for queuing up the display commands between the threads even with index with some modifications.

I can share the code in my fork repo in few days if needed for reference. Not ready for a pull request yet though 😄

[alexwbaule]

Yes @sambasan, if you get the code and the stream file that i post here , and run it, you see only the changed pixels been update.
The stream i get from the original app, using wireshark.

20230530_124822.mp4

[arthurferrai]

Are you sending exactly the same stream? I mean, it looks like it's struggling with "cleanup". Do you need to send BG pixels again? What happens if a video is playing?

[alexwbaule]

I sended exactly the same stream, you can see it on my post. What i can sended is the "Full image background" i dont know whats happens with the stream, its dont receive the "full_png_success". If i remove the background from the stream, works exactly what you see in the video.

[alexwbaule]

I build a version with support to 5 inch.

https://github.com/alexwbaule/turing-smart-screen-python/tree/support-5inch

Its UGLY, but, its work, with some bugs...

Its in my fork.

[arthurferrai]

I build a version with support to 5 inch.

https://github.com/alexwbaule/turing-smart-screen-python/tree/support-5inch

Its UGLY, but, its work, with some bugs...

Its in my fork.

I'll give a try.
Do I need to use any specific parameters to run simple_program.py?

[alexwbaule]

hi @arthurferrai

I dont edit the simple_program.py, i create a new lcd_comm_rec_c.py , and change somethigs to work with this modifications.

So, if you call main.py, it will work.

[alexwbaule]

@arthurferrai my branch has a lot of debug info, some stdout writes to see the hex... you can comment this itens.
And im build it in a Linux...

[arthurferrai]

Just to note: I'm in Windows and using correct COM port.
Do I need any special configurations? Script seems to connect but nothing happens.

[arthurferrai]

looking through your repo, it looks like I'm looking for wrong branch:

[alexwbaule]

You can pull from my repo now, i add in configure.py and simple-program.py the 5inch item.

Please, use the LandscapeEarth_5Inch theme, all 3.5 themes will not work.

[arthurferrai]

I'll assume I need to change to support-5inch branch.
When I try on that branch, it gives me the following error:

01/06/2023 17:54:42 [INFO] Loading theme LandscapeEarth_5Inch from res/themes/LandscapeEarth_5Inch/theme.yaml
Traceback (most recent call last):
  File "D:\turing-smart-screen-python\main.py", line 64, in <module>
    import library.scheduler as scheduler
  File "D:\turing-smart-screen-python\library\scheduler.py", line 28, in <module>
    import library.stats as stats
  File "D:\turing-smart-screen-python\library\stats.py", line 34, in <module>
    from library.display import display
  File "D:\turing-smart-screen-python\library\display.py", line 23, in <module>
    from library.lcd.lcd_comm_rev_c import LcdCommRevC
ModuleNotFoundError: No module named 'library.lcd.lcd_comm_rev_c'

Looks like lcd_comm_rev_c is missing from repo.

[alexwbaule]

My bad... its there now, you can pull

[alexwbaule]

[alexwbaule]

@arthurferrai , change to this "support-5-inches-device" branch.

All 3.5 themes are imcompatible, the sizes dont match.

This repo has 2 themes that support the 5 inch.

[arthurferrai]

Is it working/support-5-inches-device?
It looks like the theme is not committed.

[alexwbaule]

Done... pull please.
The "simple-program.py¨ is ok too.

[arthurferrai]

Right, I tried both main.py and simple-program.py.
Both seems to increase render counter and don't show traceback errors.
But nothing is being rendered on screen.

I think it might be an issue on trying to send data to serial port.

[alexwbaule]

Lets update the thread.

I "finished" the implementation. (Tested on Linux and Windows).

What is working ?

• Using a "static background" theme. (i dont implement the video yet).
• All Orientations

What is missing ?

• Get Storage, upload, play file, etc.

There is some catchs.

You can edit any theme, just change the image to 480 x 800 or 800 x 480. If you try to use the 3.5 image, the image will not be showed.

I merged in my "master" branch, in my repo.

https://github.com/alexwbaule/turing-smart-screen-python

With this "status", i can do a Pull Request to merge with this repo ? @mathoudebine

[arthurferrai]

I tried main here, and I'm still struggling to make it run correctly.
Logs don't show any error, but it looks like "Received" buffer is always empty.

[alexwbaule]

Hi @arthurferrai

You are using the correct background image ? (480 x 800 or 800 x 480)

get the last commits , with a pull ? (now i dont save images anymore.)

You see this in the log ?

02/06/2023 15:30:56 [DEBUG] Command: HELLO
02/06/2023 15:30:56 [DEBUG] HW sub-revision: SubRevision.FIVEINCH

And After this (in this order, ignoring another itens):

02/06/2023 15:32:41 [DEBUG] Received: [media_stop]
02/06/2023 15:32:45 [DEBUG] Received: [full_png_sucess]
02/06/2023 15:32:45 [DEBUG] Received: [needReSend:0|renderCnt:0]

[arthurferrai]

Nope, my log shows something like this:

02/06/2023 15:50:52 [DEBUG] Using Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
02/06/2023 15:50:52 [INFO] Tray icon has been displayed
02/06/2023 15:50:52 [DEBUG] Command: HELLO
02/06/2023 15:50:53 [WARNING] Display returned unknown sub-revision on Hello answer
02/06/2023 15:50:53 [DEBUG] HW sub-revision: SubRevision.UNKNOWN
02/06/2023 15:50:53 [INFO] Calling ScreenOn

Edit: I'm a dumb and was using wrong COM port :P

[arthurferrai]

Sorry about the mess, it's working nicely here.

[arthurferrai]

Just looking through dependencies, I think it would be nice to drop numpy and opencv dependencies (especially opencv is a HUGE library)... I might try to do that if you don't mind :D

[alexwbaule]

Go a head, i use opencv because i dont have the knowledge to work with Pil Package only, to make the images.

[arthurferrai]

I succeeded to remove opencv and numpy dependencies. the only problem is the color space conversion on this display (BGR) that is not ideal on Pillow (it works, but it's a bit inefficient, will try to improve later on).
I'm wondering the best way to send my changes: if I make a fork from here or from @alexwbaule

Another thing that I noticed is that it doesn't shut down the display on exit (on my machine, it stays on after pc shuts down).

[alexwbaule]

The shutdown is commented on the code, but its there. When you turn off the display, to start again you need to re-initialize the connection.

[arthurferrai]

The auto-detect works on your environment? Asking because I think serial_number of my unit is different from what's committed.

[alexwbaule]

I set the port directly, the AUTO doesn't work for me.
I build a PR (#252) if you want to join your code here, you can.

[arthurferrai]

Did you gave me write access to your repo? Or should I fork yours?

[alexwbaule]

I sended a invite to you , accept and you can write to the repo.

[alexwbaule]

If you write your modifications in the original branch (working/support-5-inches-device) the PR will be updated.

[arthurferrai]

Done.
@alexwbaule (and anyone else), please test to see if I did it correctly :)

[alexwbaule]

Tks @arthurferrai !

I will test it now...

[alexwbaule]

Its Working here...

For me, thats fine... the PR is open, and with our commits.

[gwendal-h]

Hello, I just added a pull request for the video support: #348
I still have to improve the performance: the refresh is a bit slow and it takes too much CPU.

[sambasan]

Looks awesome @gwendal-h. I got stuck with the video visible pixel color format but it seems you figured it out. Nice!

[gwendal-h]

Small video example :

20231005.195201.mp4

[gwendal-h]

There's 1 issue left :
Sometimes the update of the text/image freezes. And the status reply is "NeedReSend = 1" ...
I still have to investigate a bit on that. I suspect an ill-formed message.

Performance benchmark:
(the more pixels updated the longer it takes, so I'm doing the benchmark with the file "simple-program-video-mode.py")

My first commit, without any optimizations, one refresh takes about 330 ms on my computer.
2nd commit, with a few optimizations one refresh takes about 110 ms

I'm working on a big improvement so far i'm sending the full overlay bitmap + the visible pixels command to perform a refresh.
It is possible to only send the updated pixels + the visible pixels command.

The first tries are encouraging: from 100 000 bytes down to only 10 000 bytes sent on the serial port to perform 1 refresh.

[gwendal-h]

Finally fixed the freeze issue, and improved refresh time a lot :

refresh_with_numba.mp4