Sur d’autres sites (9905)

• How to get video pixel location from screen pixel location ?

  22 février 2024, par AmLearning

  Wall of Text so I tried breaking it up into sections to make it better sorry in advance

  


  The problem

  


  I have some video files that I am reading with ffmpeg to get the colors at specific pixels, and all seems well, but I just ran into a problem with finding the right pixel to input. I realized (or mistakingly believe) that the pixel location (x,y) on the screen will be different than the local pixel location so to speak of the video (ie. If I want to get pixel 50,0 of the video that will be different than my screen's pixel 50,0 because the resolutions don't match). I was trying to think of a way to convert my screen's pixel location into the "local pixel location", and I have two ideas but I am not sure if any of them is any good. Note I am currently using cmd+shift+4 on macos to get the screen coordinates and the video is playing fullscreen like in the screenshot below.

  


  Ideas

  


  


  1. If I manually measure and account for this vertical offset, would it effectively convert the screen coordinate into the "local" one ?

     


  


  2. If I instead adjust my SwsContext to put the destination height and width as that of my screen, will it effectively replace the need to convert screen coordinates to the video coordinates ?

     


  


  


  Problems with the Ideas

  


  The problems I see with the first solution are that I am assuming there is no hidden horizontal offset (or conversely that all of the width of the video is actually renderable on the screen). Additionally, this solution would only get an approximate result as I would need to manually measure the offsets, screen width, and screen height using the method I currently am using to get the screen coordinates.

  


  With the second solution, aside from the question of if it will even work, the problem becomes that I can no longer measure what the screen coordinates I want are because I can't seem to get rid of those black bars in VLC.

  


  Some Testing I did

  


  Given that if the black bars are part of the video itself, my entire problem would be fixed (maybe ?) I tried seeing if the black bars were part of the video, and when I looked at the frame data's first pixel, it was black. The problem then is that if the black bars are entirely part of the video, then why are the colors I get for some pixels slightly off (I am checking with ColorSync Utility). These colors aren't just slightly off as in wrong but it seems more that they belong to a slightly offset region of the video.

  


  However, this may be somewhat explained if ffmpeg reads right to left. When I put the top left corner of the video into the program and looked again at the pixel data in the frame for that location (location again was calculated by assuming the video location would be the same as the screen location) instead of getting white, I got a bluish color much like the glove in the top right corner.

  


  The Watered Down Code

  


      struct SwsContext *rescaler = NULL;
    rescaler = sws_getContext(codec_context->width, codec_context->height, codec_context->pix_fmt, codec_context->width, codec_context->height, AV_PIX_FMT_RGB0, SWS_FAST_BILINEAR, NULL, NULL, 0);

// Get Packets (containers for frames but not guaranteed to have a full frame) and Frames
    while (av_read_frame(avformatcontext, packet) >= 0)
    {
        
        // determine if packet is video packet
        if (packet->stream_index != video_index)
        {
            continue;
        }
        
        // send packet to decoder
        if (avcodec_send_packet(codec_context, packet) < 0)
        {
            perror("Failed to decode packet");
        }
        
        // get frame from decoder
        int response = avcodec_receive_frame(codec_context, frame);
        if (response == AVERROR(EAGAIN))
        {
            continue;
        }
        else if (response < 0)
        {
            perror("Failed to get frame");
        }
        
        // convert frame to RGB0 colorspace 4 bytes per pixel 1 per channel
        response = sws_scale_frame(rescaler, scaled_frame, frame);
        if(response < 0){
            perror("Failed to change colorspace");
        }
        // get data and write it
        int pixel_number = y*(scaled_frame->linesize[0]/4)+x; // divide by four gets pixel linesize (4 byte per pixel)
        int byte_number = 4*(pixel_number-1); // position of pixel in array
        // start of debugging things
        int temp = scaled_frame->data[0][byte_number]; // R
        int one_after = scaled_frame->data[0][byte_number+1]; // G
        int two_after = scaled_frame->data[0][byte_number+2]; // B
        int als; // where i put the breakpoint
        // end of debugging things
    }


  


  In Summary

  


  I have no idea what is happening.

  


  I take the data for a pixel and compare it to what colorsync utility says should be there, but it is always slightly off as though the pixel I was actually reading was offset from what I thought I was reading. Therefore, I want to find a way to get the pixel location in a video given a screen coordinate when the video is in fullscreen, but I have no idea how to (aside from a few ideas that are probably bad at best).

  


  Also does FFMPEG put the frame data right to left ?

  


  A Video Better Showing My Problem

  


  https://www.youtube.com/watch?v=NSEErs2lC3A

  


• ffmpeg-How to burn subtitles into the video

  28 décembre 2013, par user3141070

  Sir i want to burn subtitles into video.
  i use this command

  ffmpeg -i video.avi -vf subtitles=subtitle.srt out.avi

  but nothing is burn on video.
  My video is 30 sec.
  and my srt file is like this

  1
  
  00:00:04,700 --> 00:00:06,736
  
  where are you going so early?
  
  
  
  2
  
  00:00:06,900 --> 00:00:09,494
  
  Don't record any more messages
  
  on my alarm clock.
  
  
  
  3
  
  00:00:09 --> 00:00:11
  
  why not?
  
  
  
  4
  
  00:00:11 --> 00:00:13
  
  I'll start to think we're married
  
  or something.
  
  
  
  5
  
  00:00:14 --> 00:00:15
  
  Don't ever say that word.
  
  
  
  6
  
  00:00:15 --> 00:00:20
  
  I'll never bring you chicken soup
  
  and fuck your brains out again.
  
  
  
  7
  
  00:00:20--> 00:00:25
  
  - How's your cold?
  
  - Still there. How about yours?
  
  
  
  8
  
  00:00:25 --> 00:00:29
  
  - You definitely took my mind off it.
  
  - Really?

  so what's the problem ?

• How can I stream very long H.264 videos over HTTP without long startup times ? (or : MOOV atom too large)

  5 octobre 2012, par michael

  I want to stream videos of public meetings that are often 10 hours long. Users will generally be starting at some point in the middle of the video and jumping around frequently.

  H.264 in an MP4 container seems like the current best option for encoding streaming video (right ?). But these are large files — about 1.3GB for one file — and the metadata (MOOV atom) at the beginning of the file is itself about 40MB. If I understand correctly, clients need to download the full metadata before they're able to seek within the remainder of the file. Obviously, having to download 40MB before you can start streaming is unacceptable.

  (My tests of streaming have been with VLC and the HTML5 tag in Chrome.)

  I'm encoding the file using avconv, and am currently providing no additional settings beyond telling it which encoders to use (x264 and libfaac). I then move the metadata to the beginning of the file using qt-faststart.

  Is there a way to make the MOOV atom smaller ?

  If not, are there other strategies to use (e.g. is splitting long videos into several files something that's frequently done ? it seems like a real pain in terms of users seeking around the whole day's video) ?

  Or should I be using a different codec or container ?

  thanks !

  Here's a breakdown of the file header structure, from AtomicParsley :

  Atom ftyp @ 0 of size: 32, ends @ 32
  
  Atom moov @ 32 of size: 40157673, ends @ 40157705
  
       Atom mvhd @ 40 of size: 108, ends @ 148
  
       Atom iods @ 148 of size: 24, ends @ 172
  
       Atom trak @ 172 of size: 20156304, ends @ 20156476
  
           Atom tkhd @ 180 of size: 92, ends @ 272
  
           Atom edts @ 272 of size: 36, ends @ 308
  
               Atom elst @ 280 of size: 28, ends @ 308
  
           Atom mdia @ 308 of size: 20156168, ends @ 20156476
  
               Atom mdhd @ 316 of size: 32, ends @ 348
  
               Atom hdlr @ 348 of size: 45, ends @ 393
  
               Atom minf @ 393 of size: 20156083, ends @ 20156476
  
                   Atom vmhd @ 401 of size: 20, ends @ 421
  
                   Atom dinf @ 421 of size: 36, ends @ 457
  
                       Atom dref @ 429 of size: 28, ends @ 457
  
                   Atom stbl @ 457 of size: 20156019, ends @ 20156476
  
                       Atom stsd @ 465 of size: 147, ends @ 612
  
                           Atom avc1 @ 481 of size: 131, ends @ 612
  
                               Atom avcC @ 567 of size: 45, ends @ 612
  
                       Atom stts @ 612 of size: 6115608, ends @ 6116220
  
                       Atom stss @ 6116220 of size: 16960, ends @ 6133180
  
                       Atom ctts @ 6133180 of size: 5683824, ends @ 11817004
  
                       Atom stsc @ 11817004 of size: 28, ends @ 11817032
  
                       Atom stsz @ 11817032 of size: 4169724, ends @ 15986756
  
                       Atom stco @ 15986756 of size: 4169720, ends @ 20156476
  
       Atom trak @ 20156476 of size: 20001133, ends @ 40157609
  
           Atom tkhd @ 20156484 of size: 92, ends @ 20156576
  
           Atom mdia @ 20156576 of size: 20001033, ends @ 40157609
  
               Atom mdhd @ 20156584 of size: 32, ends @ 20156616
  
               Atom hdlr @ 20156616 of size: 45, ends @ 20156661
  
               Atom minf @ 20156661 of size: 20000948, ends @ 40157609
  
                   Atom smhd @ 20156669 of size: 16, ends @ 20156685
  
                   Atom dinf @ 20156685 of size: 36, ends @ 20156721
  
                       Atom dref @ 20156693 of size: 28, ends @ 20156721
  
                   Atom stbl @ 20156721 of size: 20000888, ends @ 40157609
  
                       Atom stsd @ 20156729 of size: 96, ends @ 20156825
  
                           Atom mp4a @ 20156745 of size: 80, ends @ 20156825
  
                               Atom esds @ 20156781 of size: 44, ends @ 20156825
  
                       Atom stts @ 20156825 of size: 9348168, ends @ 29504993
  
                       Atom stsc @ 29504993 of size: 28, ends @ 29505021
  
                       Atom stsz @ 29505021 of size: 5326296, ends @ 34831317
  
                       Atom stco @ 34831317 of size: 5326292, ends @ 40157609
  
       Atom udta @ 40157609 of size: 96, ends @ 40157705
  
           Atom meta @ 40157617 of size: 88, ends @ 40157705
  
               Atom hdlr @ 40157629 of size: 33, ends @ 40157662
  
               Atom ilst @ 40157662 of size: 43, ends @ 40157705
  
                   Atom ©too @ 40157670 of size: 35, ends @ 40157705
  
                       Atom data @ 40157678 of size: 27, ends @ 40157705
  
  Atom free @ 40157705 of size: 8, ends @ 40157713
  
  Atom mdat @ 40157713 of size: 1320096566, ends @ 1360254279