Migrate Iot.Devices.Camera to use new application naming

[mg-yolo-enterprises]

Per https://github.com/raspberrypi/rpicam-apps/blob/main/README.md the libcamera-* name has been updated to rpicam-* and "Users are encouraged to adopt the new application and library names as soon as possible".

libcamera is still working with the current version of applications distributed with Raspberry Pi OS.

Further discussion: https://forums.raspberrypi.com/viewtopic.php?t=359569

[raffaeler]

Thank you for catching this.
I just opened an issue to the relevant repository asking the correct way to detect the presence of the correct tool set.
We must ensure that any upgrade to our library does not break the compatibility of the app.

[naushir]

What about possibly using the rpicam-apps version string? The API function RPiCamAppsVersion() will return a string representing the version number in the form major.minor.patch. You can test for version > 1.4.0 when we switch to rpicam-*. Would that work for you?

[raffaeler]

I am not familiar with the RPiCamAppsVersion() API.

• What library does expose it?
• Is it available in both libcamera and rpi-* OS versions?

From the sources, I see it is a C++ library. This is a problem because C++ does not have a standard ABI. The mangled name of the function may change in future recompilation and break the library. Such APIs should be exposed using the "C" contract which always work.
Is there a "C" binding too?

Thanks

[naushir]

In libcamera-* land, this API function used to be called LibcameraAppsVersion(). So you would have to open up the rpicam-apps.so/libcamera-apps.so, and look for the symbol LibcameraAppsVersion() or RPiCamAppsVersion() with dlsym, which is not ideal. Also, this is using C++ mangling, but I can fix that easily enough to use a C contract.

Instead of this, how about using apt-policy to check the package name:

$ apt policy rpicam-apps 2>/dev/null | grep 'Candidate:' | awk '{print $2}'
1.5.3-1

Would that be more suitable?

[raffaeler]

Unfortunately Libcamera C++ bindings are causing a lot of problems to everyone producing a library in any language.

Running the apt-policy involves running a bash command from the library which may slow down the operatings requesting to open a channel with the camera. I would prefer something different, if possible.

The problem is that this operation may eventually be run many times when someone is using the library to open/close the camera multiple times.
While we can create a sort of detection call at the beginning, this would break our current API.

[naushir]

Perhaps you can explain how you are using the library .so file in more detail? This would help reach a better solution if possible.

[raffaeler]

@naushir sure, let me explain.
In the legacy configuration, we could easily use the libraries to bind .NET (C#) to the .so libraries and invoke the APIs to acquire a photo or a video stream.

When Raspbian transitioned to the libcamera stack, I had a long conversation with Kieran Bingham to understand how to use the Libcamera library. You can see a (long) summary of the problems here: #1875

But let me write a TLDR here:
Libcamera is a C++ library only and there is no "C" binding. This prevents any reliable compilation to make interop from any language. Every time a compilation happens (for example when a Raspbian version is published), a different C++ compiler version may be used and this may produce different mangled names for the C++ exports. This automatically breaks anyone relies on mangled names.

BTW the Python bindings provided by the Raspberry PI foundation do not suffer from this problem because they are generated and shipped along with the OS. If you always recompile the binaries and the interoperability code, you can safely use the mangled names.

To overcome this problem, I created a "binding" that invokes the libcamera apps and capture the stdout/stderr. It's far from being perfect but is a great workaround to avoid the problems described above. I hardcoded the libcamera tool names because I could never image that they could change at any time.

We are not alone. The entire community around the Raspberry PI has been hit by this same problem. There are a lot of libraries that are still recommending to switch to the legacy stack for this reason. But with the latest 64 bits OS versions the legacy stack is not provided anymore and therefore all those libraries are broken.
The renaming of the tools is breaking us now, but I bet we are not alone again.

What do we need (and not just us)?

1. In the short term, we need reliable detection method to understand whether we should run the libcamera or the newer rpicam- tools
2. In the long term, a stable "C" ABI to allow the rest of the world to safely inteop with those libraries
3. A safe stable "C" ABI for the HW H.264 encoder that used to work in the legacy stack and that is no more used by the Libcamera stack (at least this is what Kieran told me).

Thanks

[naushir]

We are not alone. The entire community around the Raspberry PI has been hit by this same problem. There are a lot of libraries that are still recommending to switch to the legacy stack for this reason. But with the latest 64 bits OS versions the legacy stack is not provided anymore and therefore all those libraries are broken.

I won't go into the details as this is been discussed many times over the years, but the legacy stack was a dead end for Raspberry Pi. It's fully closed source (so no 3rd party camera support), extremely difficult to maintain because of closed binary blobs, and would not gain any advanced camera features (e.g. PDAF or HDR). Using libcamera alleviates all these issues and adds significantly more control of the imaging system to our users.

• In the short term, we need reliable detection method to understand whether we should run the libcamera or the newer rpicam- tools

I could declare RPiCamAppsVersion() as extern "C" and you can then use dlsym() to search for the presence of the symbol. If present, assume rpicam-* else use libcamera-*. Would that work?

• In the long term, a stable "C" ABI to allow the rest of the world to safely inteop with those libraries

I know the libcamera development team have plans for C bindings for the API, but cannot comment on timeframes for when this may be available. But rpicam-apps will remain a C++ API even if/when a C API for libcamera is available. We don't have the engineering resources to rework this ourselves.

• A safe stable "C" ABI for the HW H.264 encoder that used to work in the legacy stack and that is no more used by the Libcamera stack (at least this is what Kieran told me).

This is already available as the codecs go through a standard V4L2 memory-to-memory interface. You can see the interface in rpicam-apps here.

[raffaeler]

I won't go into the details as this is been discussed many times over the years

Sure, I just wanted to start from a common acknowledgement. I fully share the reasons for the migration, but my personal opinion is that the choice to have a C++ library only is causing a huge damage to the ecosystem.

I could declare RPiCamAppsVersion() as extern "C" and you can then use dlsym() to search for the presence of the symbol. If present, assume rpicam-* else use libcamera-*. Would that work?

Yes, this would be a great way to identify the correct stack.
Does the older releases of the OS (the one with libcamera) have this library?
If yes, will they get this export too by apt updating the OS?

I know the libcamera development team have plans for C bindings for the API, but cannot comment on timeframes for when this may be available. But rpicam-apps will remain a C++ API even if/when a C API for libcamera is available. We don't have the engineering resources to rework this ourselves.

Kieran told me this as well a long time ago telling also that is not a priority. This means that it will take years before seeing something like that. Among the others, the rust, telephony and robotics communities are also struggling on this, but nothing happened. Libcamera is too vast to think that anyone else from the community will start wrapping the C++ library, not to count on the efforts required to keep it updated over the libcamera future changes.

This is already available as the codecs go through a standard V4L2 memory-to-memory interface. You can see the interface in rpicam-apps here.

I've see that in the past, but the VL42 compatibility layer for libcamera does not work as I expected (I already tried that) and I am not sure how to pipe the video into the encoder to get the H.264 stream out of a libcamera stack.
Also, I am not sure that VL42 works when libcamera has been switched on (my ignorance BTW).
Could you please elaborate on this?

Thank you very much!

[naushir]

Does the older releases of the OS (the one with libcamera) have this library?

The updates will only be available on Bookworm and later. So you will have to do a dlsym to check if the symbol exists.

I've see that in the past, but the VL42 compatibility layer for libcamera does not work as I expected (I already tried that) and I am not sure how to pipe the video into the encoder to get the H.264 stream out of a libcamera stack.
Also, I am not sure that VL42 works when libcamera has been switched on (my ignorance BTW).
Could you please elaborate on this?

There may be some confusion here. libcamera does indeed have a compatibility layer, but my link in the comment above points to a V4L2 H.264 codec API used by rpicam-apps. This is entirely separate and unrelated to the libcamera V4L2 compatibility layer. Note that libcamera only deals with uncompressed image streams. Compression (either still image/JPEG, or video/H.264) happens outside the libcamera domain - in our case at the rpicam-apps application layer.

[raffaeler]

The updates will only be available on Bookworm and later. So you will have to do a dlsym to check if the symbol exists.

Got it:

• looking for the rpicam-apps.so/libcamera-apps.so file and loading it
• dlsym to check for the symbol that you will tell us

At this point, beyond just seeing if the symbol exists, do you see any value in getting the version number? I am currently using the libcamera command line. If anything is different there, I should understand it from the version number, right?

Compression (either still image/JPEG, or video/H.264) happens outside the libcamera domain - in our case at the rpicam-apps application layer.

Right.
Please correct me if this is correct: even with libcamera support, I can still use VL42 interfaces to access the H.264 decoding, without loading any library. Correct?
Does the H.264 code you linked me have any "C" exports so that I can use the encoder without VL42? Would this make any sense?

Thanks!

[naushir]

At this point, beyond just seeing if the symbol exists, do you see any value in getting the version number? I am currently using the libcamera command line. If anything is different there, I should understand it from the version number, right?

The version number is probably not useful for what you want to test.

Please correct me if this is correct: even with libcamera support, I can still use VL42 interfaces to access the H.264 decoding, without loading any library. Correct?

For V4L2 H.264 encoding that's correct, it's completely unrelated to libcamera.

Does the H.264 code you linked me have any "C" exports so that I can use the encoder without VL42? Would this make any sense?

It does not, but it can be used as simple example of how you can drive the video encoder through the V4L2 API. It's not meant to be called externally from outside rpicam-apps.

[raffaeler]

Thanks for all the answers.
May I ask you to advise when the updated library with the new "C" api will be available?

[naushir]

The code is already checked into the repo, but we don't have a target date yet for a new package release. Hopefully it will happen towards the end of the month.

[raffaeler]

To wrap-up, this is the draft of the detection algorithm.
@naushir could you please take a look at it and tell me if you see chances for fixing/improving/simplifying it? Thanks!

We'll have to discuss in the triage which part is worth running every time in the binding, and what else we can do to avoid wasting time.
In future releases, I would stick on the upcoming symbol RPiCamAppsVersion only and drop the rest.

Detection algorithm

• Read the Distributor ID using lsb_release
• If it is not Raspbian use rpicam- tools and exit

Camera stack detection:

• Look for the /boot/config.txt
  • If it does not exist, look for the /boot/firmware/config.txt
    • If it does not exist, use rpicam- tools and exit
• Extract the camera_auto_detect presence/value in config.txt:
  • If the output is 0 or there is no output or the output is not a number, the value is 0
  • Otherwise read the last number: 0 for legacy stack, 1 for libcamera stack
• If it is 0, use theraspi tools and exit

Libcamera tools detection:

• Read the Release of the OS (Raspbian does not include the '.' but we can parse it as a decimal number so that it fits the Ubuntu style)
• If the release is <= 11 (BullsEye), use libcamera- tools and exit
• If the release is > 12 (Bookworm), use rpicam- tools and exit

Libcamera tools detection on Bookworm

• Try loading in memory the library rpicam-apps.so and the RPiCamAppsVersion symbol. If it can be found , use the rpicam- tools and exit.
• Use the libcamera- tools

[naushir]

Seems reasonable!

[raffaeler]

@naushir sorry, one more question ...
is there a chance for a user to have a Bookworm version that only have only the libcamera and not rpicam tools?

[raffaeler]

@mg-yolo-enterprises can you confirm that you are able to use the camera binding on Bookworm with the current release of the dotnet/iot libraries?

I had the opportunity to access a RPi5 and I see the symbolic links to the rpicam-* tools which should avoid any breaking changes.

$ ls /usr/bin/lib* -la
lrwxrwxrwx 1 root root     12 Jun 17  2024 /usr/bin/libcamera-hello -> rpicam-hello
lrwxrwxrwx 1 root root     11 Jun 17  2024 /usr/bin/libcamera-jpeg -> rpicam-jpeg
lrwxrwxrwx 1 root root     10 Jun 17  2024 /usr/bin/libcamera-raw -> rpicam-raw
lrwxrwxrwx 1 root root     12 Jun 17  2024 /usr/bin/libcamera-still -> rpicam-still
lrwxrwxrwx 1 root root     10 Jun 17  2024 /usr/bin/libcamera-vid -> rpicam-vid
-rwxr-xr-x 1 root root    836 Jun 17  2024 /usr/bin/libcamerify
-rwxr-xr-x 1 root root  68984 Jan 25  2023 /usr/bin/libinput
-rwxr-xr-x 1 root root  15778 Nov 25  2023 /usr/bin/libnetcfg
-rwxr-xr-x 1 root root 136310 Apr  9  2024 /usr/bin/libtoolize

[naushir]

is there a chance for a user to have a Bookworm version that only have only the libcamera and not rpicam tools?

I don't think this is possible.