opengl es - Decoding and Rendering Video on Android -

what need decode video frames , render frames on trapezoidal surface. i'm using android 2.2 development platform

i'm not using mediaplayer service since need access decoded frames.

here's have far:

• i using stagefright framework extract decoded video frames.
• each frame converted yuv420 rgb format
• the converted frames copied texture , rendered opengl surface
• note using processing , not using opengl calls directly.

so problems are

• i can decode mp4 files stagefright
• the rendering slow, around 100ms 320x420 frame
• there no audio yet, can render videos still don't know how synchronize playing of audio frames.

so questions...

• how can support other video formats? shoud use stagefright or should switch ffmpeg?
• how can improve performance? should able support @ least 720p?
• should use opengl calls directly instead of processing? improve performance?
• how can sync audio frames during playback?

adding other video formats , codecs stagefright

if have parsers "other" video formats, need implement stagefright media extractor plug-in , integrate awesome player. if have omx components required video codecs, need integrate them omxcodec class. using ffmpeg components in stagefright, or using ffmpeg player instead of stagefright not seem trivial. if required formats available in opencore, can modify android stack opencore gets chosen formats. need port logic of getting yuv data opencore. (get dirty mios)

playback performance

the surface flinger, used normal playback uses overlay rendering. provides around 4 - 8 video buffers (so far have seen). can check how many different buffers getting in open gl rendering. increasing buffer improve performance. also, check time taken yuv rgb conversion. can optimize or use opensource library improve performance. open gl not used video rendering (known graphics). not sure on performance.

audio video sync

audio time used reference. in stagefright, awesome player uses audio player playing out audio. player implements interface providing time data. awesome player uses rendering video. video frames rendered when presentation time matches of audio sample being played.

shash