aKademy 2006: Phonons in Solids

Note: I cannot get fish:// working correctly through this network (connection breaks everytime i try to upload something) so i could not upload the images mentioned in the text. So you have to live without those images. - Sorry.

This talk was held by Matthias Kretz (phonon) and Kevin Ottens (solid).

First the explained the common architecture which they both use. After that the explained solid and phonon further.

Common Achitecture Pattern

Requirements:

• Cross-project collaboration ("multimedia is not our business")
• Release cycles and Binary compatiblity (other projects work for fun to, dont want to force our own cycles and rules)
• Flexilibilty: provide choice to users and distrubuors, switch subsystems on the fly
• Portability: new porting concerns (non-free platform hint hint windows) allow to do it

Solution: Frontend/Backend split (Loose coupling with other projects. Gain portability, flexibility and binary compatibility)

General Pattern: (see image)

Allows binary compatible frontend classes

Even live backend switching will be possible: frontend is signaled to switch, frontend tells all frontend objects to save state and delete their backend object, then load the new backend and reload the state with the state objects.

The question that remains is: How should this all be impelemented?

Interface Based Approach

Pros: Enforce compile time checking, easy to document, fast method calls
Cons: BC concerns on the backend side, multiple inheritance, several pointers to the backend class, you cant have a qwidget in the inheritance tree

Introspection based Approach

Pros: No BC concnerns, no need to maintain two sets of classes (frontend & interfaces), free to partially implement a backend class
Cons: no compile time checking, requires more work for explaining backend writing (no interface classes to document with doxygen, slower method calls (invokeMethod() has overhead with string parsing/comparision)

Qt to the Rescue

(image)
Declaring and Implementing Q_INTERFACES
Uses the Q_DECLARE_INTERFACE makro.

The frontend class has a QObject * member m_iface pointing to the backend object. (image?)

Solid

Why?
To fix the current situation. In KDE3 we have:

• different hardware discovery (media manager, medianotifier)
• network managment (knetworkmanager, kwifimanager)
• power management (kpowersave, klaptop, sebas' powermanager...)

This sucks becaue it is so diverse.
New use cases are now. bluetooth is already here, hardware is much more modular(pluggable) (usb/pcmcia,hotswap, etc.)

Solid will have several domains: hardware discovery, power & network managment.
And then solid has policy agents: knetworkmanager mediamanger already there. They are resoponsible to interact with the user and provide a library to access the hardware.

From solid's Point of View:

• Report as many hardware facts as possible
• no device interaction
• storage is the only exception

Domain specific interaction should be done in domain specifi code (playing music > phonon, printer > kdeprint, etc)

Hardware discovery (image)

Following some examples how easy it is to use it.

Powermanagement has one central class: PowerManager this takes care of all things related to powermanagment. (Throttle the CPU, darken the display, spin down the harddisk, etc)
It has useful signals signaling when the batterystate changed or something else

Hardware Discovery works in three levels:

• First: DeviceManager, subsystem, device listing, creates Device
• Second: Device: Parent /Child relationship, basic informations, is factory for Capabilites
• Third. Capabalities, provide specific informations about the capabilities of a device

Current state:
Hardware discovery and network management is almost done, a few features for system statistics, need more applications using it.
Policy agents: porting medianotifier, knetworkmanager and friends (target for the coding marathon)
Backend: everything required for linux like systems is done, more backends need to be written for other systems

Phonon

Motivation: (the users perspective)

• A user should be able to play back media without any configuration
• "Power users" want great flexibility
• additional multimedia hardware (think usb-headset) should be available to all applications without any further steps
• Users need to decide what device use for what purpose/program

(the developers perspective)

• Qt/KDE style api
• developers need APIs that are straight forward and easy to use
• applications need multimedia API which also works on completly different platforms (think MacOsX, Windows)
• ABI/API changes in multimedia framworks should not hinder KDE to use the newest mediaframework

Core classes (image)

Phonon Audio player: A very easy to use class called AudioPlayer, that provides enough features to be used by Juk. It provides methods to play() a KURL or prelod a file and then play it back later with smaller latency. Pause() Stop() and Seek() methods provide all that is needed for basic audio playing support in applications.
Example: (image 2228)


The VideoPlayer class functions nearly the same, the only difference is, that it is a QWidget and you can embed it everywhere you like.
Example: (image 2229)

But modern audo players want more. (gapless playback, crossfades, equalizer, brightness controls, audio visualizations, etc).
For this use cases, the PhononMediaQueue comes into play. It allows to preload the next file (for crossfades) and use the Effects API to do other things.

Currently there are 3 and a half backend implementations: NMM, Xine, avKode and Fake.
NMM is the one which is likely to be most feature complete, because it is the only to provide recording for the AvCapture part of phonon. I also has already support to stream data over KIO so any valid KUrl may be used for media playback.
Xine is the one which gets out results really quickly, but will never be as complete since recording is missing completely. avKode was a Sommer of Code project and is based on ffmpeg and mencode(?)

In the following Matthias explained how the xine backend is implemented (as it is him who implements it) and shows the challenges he faced.


Todo list: (existing backends can only do simple playback, no more)

• Implement Xine::ByteStream for KIO Urls
• Xine video breaks because it wants XThreads
• NMM nees a VideoWidget implementation
• AvCapture API needs to be finished implemented
• Effects (EQ, Fader, Compressor, etc)
• Phonon-Gstreamer anyone?
• KIO Seeking in MediaObject
• (good) user interface for phonon configuration
• Network API: VoIP
• DVD/TV support, chapters
• OSD (video overlay)
• device listing (how to get alsa device list?)

As you see, there is still lots of things to do, so please consider joining the efforts!


Personal Comment: I really appreciate the work the two guys do, and I am full of hope that for KDE4 the "plug and play" for hardware and audio will "just work", and provide a convienient, unified and straigth forward interface to the user.