It sounds like you have 2D graphics showing over your video.

Here is a simple example of using the API:
http://www.libsdl.org/tmp/slvideo.zip

If you replace slvideo.cpp with slvideo_OpenGL.cpp you can see how to mix other graphics with the video.

Originally posted by slouken:

It sounds like you have 2D graphics showing over your video.

Here is a simple example of using the API:
http://www.libsdl.org/tmp/slvideo.zip

If you replace slvideo.cpp with slvideo_OpenGL.cpp you can see how to mix other graphics with the video.

I see a lot of OpenGL related stuff in this example code, how would this apply if I were just using the accelerated 2D rendering APIs in SDL?

Originally posted by Hello World:

Originally posted by slouken:

Here is a simple example of using the API:
http://www.libsdl.org/tmp/slvideo.zip

No longer compiles.

slvideo.cpp:50:82: error: too few arguments to function 'CSLVideoStream* SLVideo_CreateStream(CSLVideoContext*, ESLVideoFormat, int)'
CSLVideoStream* stream = SLVideo_CreateStream(context, k_ESLVideoFormatH264);
^
In file included from slvideo.cpp:29:0:
steamlink-sdk-master/rootfs/usr/include/SLVideo.h:187:41: note: declared here
extern SLVIDEO_DECLSPEC CSLVideoStream *SLVideo_CreateStream( CSLVideoContext *pContext, ESLVideoFormat eFormat, int bLowLatency );

What does bLowLatency do? What is it used for?

In low latency mode, it seems not to do much (if any at all) caching...any data you send is immediately presented. In normal mode it will happily take as much data as you give it and cache it before playing it. I don't really know how one is supposed to use the low latency mode, I'd assume it would require supplying a custom stream instead of one from a standard source.

Originally posted by Hello World:

// If bLowLatency is non-zero, the stream will be optimized for low latency display and
// will only support I and P frames.

What is the difference between low latency display and normal display? How do I know if I should have this enabled or not? I and P frames?

https://en.wikipedia.org/wiki/Video_compression_picture_types

Does it depend on the video that is being streamed? How do I know if the video I am streaming uses I, P or B frames?

This is so confusing.

Yes. It depends on the video being streamed.

As far as I understand, pretty much all standard h264 streams contain I, P, and B frames, so its safe bet that unless what your streaming from produces non-standard streams for some reason, it has all 3 types of frames.

Probably best to just set bLowLatency to 0.

Originally posted by Hello World:

Originally posted by rstat1:

I've gotten all the correct URLs and am using libav to get video (seemingly without error) to passi off to SLVideo to decode....

How are getting the video? Could you show an example?

Most of how I get the proper URLs is 100% specific to Twitch (since my app is a twitch player) so probably wouldn't be useful to you unless you're also making a twitch player.

I would strongly suggest using something like FFMPEG (or, rather libavcodec and friends) to correctly read your files, instead of doing it they way you're showing. Any of the tutorial/example code for FFMPEG would show you a close approximation of how I'm getting the actual video data. Though if you do actually use FFMPEG and look at its examples you'll want to ignore anything about decoding because that part is what SLVideo/SLAudio does. For audio you'll likely also need to do some conversion because SLAudio requires a specific frequency and data type to work properly.

Originally posted by Hello World:

Originally posted by rstat1:

Most of how I get the proper URLs is 100% specific to Twitch (since my app is a twitch player) so probably wouldn't be useful to you unless you're also making a twitch player.

I see, and did you ever make any progress with your Twitch app? I am really only interested in watching live stream without the audio delay the Kodi app gives you. Perhaps this would be possible with SLVideo.

Yes I did manage to make a fully functional twitch player, and I've been using it for a while now. I haven't fully solved the audio delay issue though...but it is 1000x better than the kodi twitch plugin is.

Originally posted by Hello World:

You said before that you could only see the last frame, I am curious, how did you start your app? I have noticed that starting an app over an SSH connection and starting it from the Steam Link menu does not always show you the same thing on the screen.

The main problem I had with "only seeing the last frame" was related to how I was setting up the SDL window for drawing a UI over the video. Turns out, that window needs a transparent background. The frame I was seeing was only appearing when I closed my app.

As far as launching my app, I just looked at the examples in the steamlink sdk, and how they were started and modeled my stuff off of that. Here's an example (starting at Line 23 to Line 26):

https://github.com/ValveSoftware/steamlink-sdk/blob/master/examples/sdl/build_steamlink.sh#L23

The only issues I've had as far launching from SSH vs the start screen is that audio didn't seem to work when I launched via SSH.

Originally posted by Hello World:

Originally posted by rstat1:

I would strongly suggest using something like FFMPEG (or, rather libavcodec and friends) to correctly read your files, instead of doing it they way you're showing.

Hmm, in the example Sam provided, he is getting all files in the directory /home/steam/packets. He then reads each entire file from that folder into a buffer, which he gives directly, without modification, to the function SLVideo_WriteFrameData.

This gave me the impression you could just pass any data from a video stream to SLVideo_WriteFrameData and it would play the video. But that is not how it works?

I find it interesting that I can see half the video, the top half, I am obviously doing something wrong, I just don't know what.

I'd imagine the video data being used in the sample code was just straight h264 packets, which is perfectly fine for a test case. But in a normal video file you can't just open it like a normal file and pass whatever data you get to the video decoder since it'll be a combo of both video and audio in different formats. You'll need something that can properly parse/read mpeg4 to get to the data that you can use with SLVideo_WriteFrameData.

Originally posted by rstat1:

Could someone who's smarter than me explain (or better yet show a simplifed example of) how the SLVideo API is supposed to be used?

I've an app I'm building for personal use that plays twitch streams....I've gotten all the correct URLs and am using libav to get video (seemingly without error) to passi off to SLVideo to decode....problem is the video never shows on the screen. I don't get a single error through out all of this. I only ever see a single frame AFTER my app closes.

I'm trying to do something similar, creating a simple app to feed a video stream from somewhere to the SLVideo decoder. I'm not having any issues with getting output on the screen, but I keep having problems with SLVideo complaining about stream macroblock errors.

I'm demuxing with libavformat and then sending the data (through av_interleaved_write_frame()) to the avformat raw h264 "muxer" (mainly so it can apply the h264_mp4toannexb bitstream filter).

I'm using avio_alloc_context() so that all the data from the h264 muxer gets sent to a custom write_packet() function which calls SLVideo_BeginFrame(), SLVideo_WriteFrameData() and SLVideo_SubmitFrame().

Using this method, my program works for about 10 seconds (on a local video) before SLVideo errors on BeginFrame(). My best guess is that avformat is not giving write_packet() a single continuous frame at that point or some of the packets are out of order.

Any suggestions on a way to fix this or ideas on what I could be doing wrong?

Originally posted by Hello World:

Originally posted by /dev/player0:

I'm not having any issues with getting output on the screen, but I keep having problems with SLVideo complaining about stream macroblock errors.

Using this method, my program works for about 10 seconds (on a local video) before SLVideo errors on BeginFrame()

In the SLVideo.h header file it says:

//-------------------------------------------------------------------------------------------------- // Start decoding a frame of the given size // This function returns 0 on success or -1 if there is a macroblock error or the decoder can't // keep up with the data rate. //-------------------------------------------------------------------------------------------------- extern SLVIDEO_DECLSPEC int SLVideo_BeginFrame( CSLVideoStream *pStream, int nFrameSize );

I am only guessing now, but maybe your are giving that function too large data chunks to deal with, or you are calling it too frequently.

Hmmm... I'm not so sure. SLVideo prints out a message specifically saying it's a macroblock error and prints another message when it receives too much data.

Well I think I've solved my problem. The key was to set the 'direct' flag on my AVIOContext.

This means that when I write demuxed data to the h264 muxer it doesn't try to split packets up and buffer them to suit the size of the buffer avio_alloc_context() asks you to provide. The data is just delivered straight to my write_packet() function. You can just pass NULL to avio_alloc_context() in place of a buffer.

I did come across a few problems after solving this issue where SLVideo would not accept a new frame because it was "out of space". Sleeping until SLVideo_GetQueuedVideoMS() returns less than ~500ms in my demuxing loop and sleeping for a few hundred milliseconds when SLVideo_BeginFrame() returns an error (and then retrying) seems to fix this.

Now for audio...

Originally posted by /dev/player0:

Well I think I've solved my problem. The key was to set the 'direct' flag on my AVIOContext.

This means that when I write demuxed data to the h264 muxer it doesn't try to split packets up and buffer them to suit the size of the buffer avio_alloc_context() asks you to provide. The data is just delivered straight to my write_packet() function. You can just pass NULL to avio_alloc_context() in place of a buffer.

I did come across a few problems after solving this issue where SLVideo would not accept a new frame because it was "out of space". Sleeping until SLVideo_GetQueuedVideoMS() returns less than ~500ms in my demuxing loop and sleeping for a few hundred milliseconds when SLVideo_BeginFrame() returns an error (and then retrying) seems to fix this.

Now for audio...

I ran in to similar issues, and paused my decode loop until the queued video was at or below 3.5 seconds...so bascially the same thing you did :P

As for audio, the steam link's decoder expects a very specific format (and seems to only support that one format) so you're gonna have to massage your data a little bit before sending it off to be played. Also I found using SDL's audio functions much easier than trying to talk directly to SLAudio (SDL does all the work of calling SLAudio for you).

Originally posted by rstat1:

Originally posted by /dev/player0:

Well I think I've solved my problem. The key was to set the 'direct' flag on my AVIOContext.

This means that when I write demuxed data to the h264 muxer it doesn't try to split packets up and buffer them to suit the size of the buffer avio_alloc_context() asks you to provide. The data is just delivered straight to my write_packet() function. You can just pass NULL to avio_alloc_context() in place of a buffer.

I did come across a few problems after solving this issue where SLVideo would not accept a new frame because it was "out of space". Sleeping until SLVideo_GetQueuedVideoMS() returns less than ~500ms in my demuxing loop and sleeping for a few hundred milliseconds when SLVideo_BeginFrame() returns an error (and then retrying) seems to fix this.

Now for audio...

I ran in to similar issues, and paused my decode loop until the queued video was at or below 3.5 seconds...so bascially the same thing you did :P

As for audio, the steam link's decoder expects a very specific format (and seems to only support that one format) so you're gonna have to massage your data a little bit before sending it off to be played. Also I found using SDL's audio functions much easier than trying to talk directly to SLAudio (SDL does all the work of calling SLAudio for you).

Ah... I'm looking at libswresample right now but if it's as awkward as it looks like it could be to get working, maybe SDL will be the way to go.

Originally posted by Hello World:

Guys could you share your source code? Sharing is caring.

I'll probably put my code up on GitHub when I have at least somewhat in a working state :P

Originally posted by Hello World:

Guys could you share your source code? Sharing is caring.

I suppose I could share some of my core player code since you asked so nicely :P

https://gist.github.com/rstat1/7c7918fdfd707f409ae739eac0cdb7ab

There ya go. I've included the important parts of my player. Hopefully you get something useful from it :D