Recherche avancée

Sur d’autres sites (10218)

• FFmpeg and reserved color primaries [closed]

  21 janvier, par Yoz

  I am trying to get thumbnails from a hevc video downloaded from https://github.com/stashapp/stash/issues/4124#issuecomment-1720057183 and it works with most recent ffmpeg 7.1 (installed via homebrew on mac) printing :

  


  ffmpeg -i input.mp4 -frames:v 1 out.jpg


  


  ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
  built with Apple clang version 16.0.0 (clang-1600.0.26.4)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.1_4 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.100 / 61. 19.100
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[hevc @ 0x134f07530] VPS 0 does not exist
[hevc @ 0x134f07530] SPS 0 does not exist.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp42iso2mp41
    creation_time   : 2023-09-14T19:46:05.000000Z
    encoder         : HandBrake 1.5.1 2022011000
  Duration: 00:01:26.05, start: 0.000000, bitrate: 231 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/reserved/bt709), 648x648 [SAR 1:1 DAR 1:1], 188 kb/s, 30 fps, 30 tbr, 90k tbn (default)
      Metadata:
        creation_time   : 2023-09-14T19:46:05.000000Z
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 36 kb/s (default)
      Metadata:
        creation_time   : 2023-09-14T19:46:05.000000Z
        handler_name    : Mono
        vendor_id       : [0][0][0][0]
[hevc @ 0x1358065c0] VPS 0 does not exist
[hevc @ 0x1358065c0] SPS 0 does not exist.
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'out.jpg':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp42iso2mp41
    encoder         : Lavf61.7.100
  Stream #0:0(und): Video: mjpeg, yuv420p(pc, bt709/reserved/bt709, progressive), 648x648 [SAR 1:1 DAR 1:1], q=2-31, 200 kb/s, 30 fps, 30 tbn (default)
      Metadata:
        creation_time   : 2023-09-14T19:46:05.000000Z
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.19.100 mjpeg
      Side data:
        cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
[image2 @ 0x134f16080] The specified filename 'out.jpg' does not contain an image sequence pattern or a pattern is invalid.
[image2 @ 0x134f16080] Use a pattern such as %03d for an image sequence or use the -update option (with -frames:v 1 if needed) to write a single image.
[out#0/image2 @ 0x134f10480] video:5KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame=    1 fps=0.0 q=5.1 Lsize=N/A time=00:00:00.03 bitrate=N/A speed=4.07x   


  


  however, when I use custom compiled ffmpeg.wasm it fails with :

  


  ffmpeg version N-118050-ga518b5540d Copyright (c) 2000-2024 the FFmpeg developers
  built with emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.73 (ac676d5e437525d15df5fd46bc2c208ec6d376a3)
  configuration: --target-os=none --arch=x86_32 --enable-cross-compile --enable-version3 --enable-zlib --enable-libaom --disable-encoder=libaom_av1 --enable-libopenh264 --enable-libkvazaar --enable-libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libopus --enable-libwebp --enable-libsvtav1 --enable-librubberband --disable-x86asm --disable-inline-asm --disable-stripping --disable-programs --disable-doc --disable-debug --disable-runtime-cpudetect --disable-autodetect --extra-cflags='-O3 -flto -I/ffmpeg-wasm/build/include -pthread -msimd128' --extra-cxxflags='-O3 -flto -I/ffmpeg-wasm/build/include -pthread -msimd128' --extra-ldflags='-O3 -flto -I/ffmpeg-wasm/build/include -pthread -msimd128 -L/ffmpeg-wasm/build/lib' --pkg-config-flags=--static --nm=emnm --ar=emar --ranlib=emranlib --cc=emcc --cxx=em++ --objcc=emcc --dep-cc=emcc --enable-gpl --enable-libx264 --enable-libx265
  libavutil      59. 49.100 / 59. 49.100
  libavcodec     61. 26.100 / 61. 26.100
  libavformat    61.  9.100 / 61.  9.100
  libavdevice    61.  4.100 / 61.  4.100
  libavfilter    10.  6.101 / 10.  6.101
  libswscale      8. 12.100 /  8. 12.100
  libswresample   5.  4.100 /  5.  4.100
  libpostproc    58.  4.100 / 58.  4.100
[hevc @ 0x38d0000] VPS 0 does not exist
[hevc @ 0x38d0000] SPS 0 does not exist.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp42iso2mp41
    creation_time   : 2023-09-14T19:46:05.000000Z
    encoder         : HandBrake 1.5.1 2022011000
  Duration: 00:01:26.05, start: 0.000000, bitrate: 231 kb/s
  Stream #0:0[0x1](und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709/reserved/bt709), 648x648 [SAR 1:1 DAR 1:1], 188 kb/s, 30 fps, 30 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2023-09-14T19:46:05.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 36 kb/s (default)
    Metadata:
      creation_time   : 2023-09-14T19:46:05.000000Z
      handler_name    : Mono
      vendor_id       : [0][0][0][0]
[hevc @ 0x38d0300] VPS 0 does not exist
[hevc @ 0x38d0300] SPS 0 does not exist.
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x8ca0000] Unsupported input (Not supported): fmt:yuv420p csp:bt709 prim:reserved trc:bt709 -> fmt:yuv420p csp:bt709 prim:reserved trc:bt709
[vf#0:0 @ 0x3830900] Error while filtering: Not supported
[vf#0:0 @ 0x3830900] Task finished with error code: -138 (Not supported)
[vost#0:0/mjpeg @ 0x385ae40] [enc:mjpeg @ 0x3878b80] Could not open encoder before EOF
[vf#0:0 @ 0x3830900] Terminating thread with return code -138 (Not supported)
[vost#0:0/mjpeg @ 0x385ae40] Task finished with error code: -28 (Invalid argument)
[vost#0:0/mjpeg @ 0x385ae40] Terminating thread with return code -28 (Invalid argument)
[out#0/image2 @ 0x3851580] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A    
Conversion failed!
Process finished with exit code -138.


  


  I figured out the issue is color primaries prim:reserved, and the command can be updated to a working one by re-writing input primaries as following :

  


  ffmpeg -i input.mp4 -vf "colorspace=all=bt709:iprimaries=bt709" -frames:v 1 out.jpg


  


  However, I would like to compile ffmpeg.wasm so that it handles reserved primaries just like the one from homebrew.

  


  Any idea what the compiled ffmpeg.wasm is missing ?

  


• Banking Data Strategies – A Primer to Zero-party, First-party, Second-party and Third-party data

  25 octobre 2024, par Daniel Crough — Banking and Financial Services, Privacy

  Banks hold some of our most sensitive information. Every transaction, loan application, and account balance tells a story about their customers’ lives. Under GDPR and banking regulations, protecting this information isn’t optional – it’s essential.

  Yet banks also need to understand how customers use their services to serve them better. The solution lies in understanding different types of banking data and how to handle each responsibly. From direct customer interactions to market research, each data source serves a specific purpose and requires its own privacy controls.

  Before diving into how banks can use each type of data effectively, let’s look into the key differences between them :

  Data Type	What It Is	Banking Example	Legal Considerations
  First-party	Data from direct customer interactions with your services	Transaction records, service usage patterns	Different legal bases apply (contract, legal obligation, legitimate interests)
  Zero-party	Information customers actively provide	Stated preferences, financial goals	Requires specific legal basis despite being voluntary ; may involve profiling
  Second-party	Data shared through formal partnerships	Insurance history from partners	Must comply with PSD2 and specific data sharing regulations
  Third-party	Data from external providers	Market analysis, demographic data	Requires due diligence on sources and specific transparency measures

  What is first-party data ?

  

  First-party data reveals how customers actually use your banking services. When someone logs into online banking, withdraws money from an ATM, or speaks with customer service, they create valuable information about real banking habits.

  This direct interaction data proves more reliable than assumptions or market research because it shows genuine customer behaviour. Banks need specific legal grounds to process this information. Basic banking services fall under contractual necessity, while fraud detection is required by law. Marketing activities need explicit customer consent. The key is being transparent with customers about what information you process and why.

  Start by collecting only what you need for each specific purpose. Store information securely and give customers clear control through privacy settings. This approach builds trust while helping meet privacy requirements under the GDPR’s data minimisation principle.

  What is zero-party data ?

  

  Zero-party data emerges when customers actively share information about their financial goals and preferences. Unlike first-party data, which comes from observing customer behaviour, zero-party data comes through direct communication. Customers might share their retirement plans, communication preferences, or feedback about services.

  Interactive tools create natural opportunities for this exchange. A retirement calculator helps customers plan their future while revealing their financial goals. Budget planners offer immediate value through personalised advice. When customers see clear benefits, they’re more likely to share their preferences.

  However, voluntary sharing doesn’t mean unrestricted use. The ICO’s guidance on purpose limitation applies even to freely shared information. Tell customers exactly how you’ll use their data, document specific reasons for collecting each piece of information, and make it simple to update or remove personal data.

  Regular reviews help ensure you still need the information customers have shared. This aligns with both GDPR requirements and customer expectations about data management. By treating voluntary information with the same care as other customer data, banks build lasting trust.

  What is second-party data ?

  

  Second-party data comes from formal partnerships between banks and trusted companies. For example, a bank might work with an insurance provider to better understand shared customers’ financial needs.

  These partnerships need careful planning to protect customer privacy. The ICO’s Data Sharing Code provides clear guidelines : both organisations must agree on what data they’ll share, how they’ll protect it, and how long they’ll keep it before any sharing begins.

  Transparency builds trust in these arrangements. Tell customers about planned data sharing before it happens. Explain what information you’ll share and how it helps provide better services.

  Regular audits help ensure both partners maintain high privacy standards. Review shared data regularly to confirm it’s still necessary and properly protected. Be ready to adjust or end partnerships if privacy standards slip. Remember that your responsibility to protect customer data extends to information shared with partners.

  Successful partnerships balance improved service with diligent privacy protection. When done right, they help banks understand customer needs better while maintaining the trust that makes banking relationships work.

  What is third-party data ?

  

  Third-party data comes from external sources outside your bank and its partners. Market research firms, data analytics companies, and economic research organizations gather and sell this information to help banks understand broader market trends.

  This data helps fill knowledge gaps about the wider financial landscape. For example, third-party data might reveal shifts in consumer spending patterns across different age groups or regions. It can show how customers interact with different financial services or highlight emerging banking preferences in specific demographics.

  But third-party data needs careful evaluation before use. Since your bank didn’t collect this information directly, you must verify both its quality and compliance with privacy laws. Start by checking how providers collected their data and whether they had proper consent. Look for providers who clearly document their data sources and collection methods.

  Quality varies significantly among third-party data providers. Some key questions to consider before purchasing :

  • How recent is the data ?
  • How was it collected ?
  • What privacy protections are in place ?
  • How often is it updated ?
  • Which specific market segments does it cover ?

  Consider whether third-party data will truly add value beyond your existing information. Many banks find they can gain similar insights by analysing their first-party data more effectively. If you do use third-party data, document your reasons for using it and be transparent about your data sources.

  Creating your banking data strategy

  

  A clear data strategy helps your bank collect and use information effectively while protecting customer privacy. This matters most with first-party data – the information that comes directly from your customers’ banking activities.

  Start by understanding what data you already have. Many banks collect valuable information through everyday transactions, website visits, and customer service interactions. Review these existing data sources before adding new ones. Often, you already have the insights you need – they just need better organization.

  Map each type of data to a specific purpose. For example, transaction data might help detect fraud and improve service recommendations. Website analytics could reveal which banking features customers use most. Each data point should serve a clear business purpose while respecting customer privacy.

  Strong data quality standards support better decisions. Create processes to update customer information regularly and remove outdated records. Check data accuracy often and maintain consistent formats across your systems. These practices help ensure your insights reflect reality.

  Remember that strategy means choosing what not to do. You don’t need to collect every piece of data possible. Focus on information that helps you serve customers better while maintaining their privacy.

  Managing multiple data sources

  

  Banks work with many types of data – from direct customer interactions to market research. Each source serves a specific purpose, but combining them effectively requires careful planning and precise attention to regulations like GDPR and ePrivacy.

  First-party data forms your foundation. It shows how your customers actually use your services and what they need from their bank. This direct interaction data proves most valuable because it reflects real behaviour rather than assumptions. When customers check their balances, transfer money, or apply for loans, they show you exactly how they use banking services.

  Zero-party data adds context to these interactions. When customers share their financial goals or preferences directly, they help you understand the “why” behind their actions. This insight helps shape better services. For example, knowing a customer plans to buy a house helps you offer relevant savings tools or mortgage information at the right time.

  Second-party partnerships can fill specific knowledge gaps. Working with trusted partners might reveal how customers manage their broader financial lives. But only pursue partnerships when they offer clear value to customers. Always explain these relationships clearly and protect shared information carefully.

  Third-party data helps provide market context, but use it selectively. External market research can highlight broader trends or opportunities. However, this data often proves less reliable than information from direct customer interactions. Consider it a supplement to, not a replacement for, your own customer insights.

  Keep these principles in mind when combining data sources :

  • Prioritize direct customer interactions
  • Focus on information that improves services
  • Maintain consistent privacy standards across sources
  • Document where each insight comes from
  • Review regularly whether each source adds value
  • Work with privacy and data experts to ensure customer information is handled properly

  Enhance your web analytics strategy with Matomo

  

  The financial sector finds powerful and compliant web analytics increasingly valuable as it navigates data management and privacy regulations. Matomo provides a configurable privacy-centric solution that meets the requirements of banks and financial institutions.

  Matomo empowers your organisation to :

  • Collect accurate, GDPR-compliant web data
  • Integrate web analytics with your existing tools and platforms
  • Maintain full control over your analytics data
  • Gain insights without compromising user privacy

  Matomo is trusted by some of the world’s biggest banks and financial institutions. Try Matomo for free for 30 days to see how privacy-focused analytics can get you the insights you need while maintaining compliance and user trust.

• My stitched frames colors looks very different from my original video, causing my video to not be able to stitch it back properly [closed]

  27 mai 2024, par Wer Wer

  I am trying to extract some frames off my video to do some form of steganography. I accidentally used a 120fps video, causing the files to be too big when i extract every single frame. To fix this, I decided to calculate how many frames is needed to hide the bits (LSB replacement for every 8 bit) and then extract only certain amount of frames. This means

  


  

  1. if i only need 1 frame, ill extract frame0.png

  


  2. ill remove frame0 from the original video

  


  3. encode my data into frame0.png

  


  4. stitch frame0 back into ffv1 video

  


  5. concatenate frame0 video to the rest of the video, frame0 video in front.

  


  


  I can do extraction and remove frame0 from the video. However, when looking at frame0.mkv and the original.mkv, i realised the colors seemed to be different.
Frame0.mkv
original.mkv

  


  This causes a glitch during the stitching of videos together, where the end of the video has some corrupted pixels. Not only that, it stops the video at where frame0 ends. I think those corrupted pixels were supposed to be original.mkv pixels, but they did not concatenate properly.
results.mkv

  


  I use an ffmpeg sub command to extract frames and stitch them

  


      def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)



  


  This code splits the video into the required number of frames. It checks if the total amount of frame bits is enough to encode the hidden text

  


  def remove_frames_from_video(self, input_video, frames_to_remove):
    if not input_video.endswith(".mkv"):
        input_video += ".mkv"

    input_video_path = os.path.join(self.here, input_video)

    # Create a filter string to exclude specific frames
    filter_str = (
        "select='not("
        + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
        + ")',setpts=N/FRAME_RATE/TB"
    )

    # Temporary output video path
    output_video_path = os.path.join(self.here, "temp_output.mkv")

    command = [
        "ffmpeg",
        "-y",
        "-i",
        input_video_path,
        "-vf",
        filter_str,
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        "-an",  # Remove audio
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Frames removed. Temporary video created at {output_video_path}")

        # Replace the original video with the new video
        os.replace(output_video_path, input_video_path)
        print(f"Original video replaced with updated video at {input_video_path}")

        # Re-add the trimmed audio to the new video
        self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")
        if os.path.exists(output_video_path):
            os.remove(output_video_path)

def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
    # Calculate the new duration based on the remaining frames
    fps = 60  # Assuming the framerate is 60 fps
    total_frames_removed = len(frames_to_remove)
    original_duration = self.get_video_duration(video_path)
    new_duration = original_duration - (total_frames_removed / fps)

    # Extract and trim the audio
    audio_path = os.path.join(self.here, "trimmed_audio.aac")
    command_extract_trim = [
        "ffmpeg",
        "-y",
        "-i",
        video_path,
        "-t",
        str(new_duration),
        "-q:a",
        "0",
        "-map",
        "a",
        audio_path,
    ]
    try:
        subprocess.run(command_extract_trim, check=True)
        print(f"Audio successfully trimmed and extracted to {audio_path}")

        # Add the trimmed audio back to the video
        final_video_path = video_path.replace(".mkv", "_final.mkv")
        command_add_audio = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-i",
            audio_path,
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_video_path,
        ]
        subprocess.run(command_add_audio, check=True)
        print(f"Final video with trimmed audio created at {final_video_path}")

        # Replace the original video with the final video
        os.replace(final_video_path, video_path)
        print(f"Original video replaced with final video at {video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")

def get_video_duration(self, video_path):
    command = [
        "ffprobe",
        "-v",
        "error",
        "-show_entries",
        "format=duration",
        "-of",
        "default=noprint_wrappers=1:nokey=1",
        video_path,
    ]
    try:
        result = subprocess.run(
            command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
        )
        duration = float(result.stdout.decode().strip())
        return duration
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while getting video duration: {e}")
        return 0.0


  


  here ill remove all the frames that has been extracted from the video

  


  def stitch_frames_to_video(self, ffv1_video, framerate=60):
    # this command is another ffmpeg subcommand.
    # it takes every single frame from data1 directory and stitch it back into a ffv1 video
    if not ffv1_video.endswith(".mkv"):
        ffv1_video += ".mkv"

    output_video_path = os.path.join(self.here, ffv1_video)

    command = [
        "ffmpeg",
        "-y",
        "-framerate",
        str(framerate),
        "-i",
        os.path.join(self.frames_directory, "frame%d.png"),
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Video successfully created at {output_video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  after encoding the frames, ill try to stitch the frames back into ffv1 video

  


  def concatenate_videos(self, video1_path, video2_path, output_path):
    if not video1_path.endswith(".mkv"):
        video1_path += ".mkv"
    if not video2_path.endswith(".mkv"):
        video2_path += ".mkv"
    if not output_path.endswith(".mkv"):
        output_path += ".mkv"

    video1_path = os.path.join(self.here, video1_path)
    video2_path = os.path.join(self.here, video2_path)
    output_video_path = os.path.join(self.here, output_path)

    # Create a text file with the paths of the videos to concatenate
    concat_list_path = os.path.join(self.here, "concat_list.txt")
    with open(concat_list_path, "w") as f:
        f.write(f"file '{video1_path}'\n")
        f.write(f"file '{video2_path}'\n")

    command = [
        "ffmpeg",
        "-y",
        "-f",
        "concat",
        "-safe",
        "0",
        "-i",
        concat_list_path,
        "-c",
        "copy",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Videos successfully concatenated into {output_video_path}")
        os.remove(concat_list_path)  # Clean up the temporary file
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  now i try to concatenate the frames video with the original video, but it is corrupting as the colors are different.

  


  this code does the other processing by removing all the extracted frames from the video, as well as trimming the audio (but i think ill be removing the audio trimming as i realised it is not needed at all)

  


  I think its because .png frames will lose colors when they get extracted out. The only work around I know is to extract every single frame. But this causes the program to run too long as for a 12 second video, I will extract 700++ frames. Is there a way to fix this ?

  


  my full code

  


  import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


class FFV1Steganography:
    def __init__(self):
        self.here = os.path.dirname(os.path.abspath(__file__))

        # Create a folder to save the frames
        self.frames_directory = os.path.join(self.here, "data")
        try:
            if not os.path.exists(self.frames_directory):
                os.makedirs(self.frames_directory)
        except OSError:
            print("Error: Creating directory of data")

    def read_hidden_text(self, filename):
        file_path_txt = os.path.join(self.here, filename)
        # Read the content of the file in binary mode
        with open(file_path_txt, "rb") as f:
            hidden_text_content = f.read()
        return hidden_text_content

    def calculate_length_of_hidden_text(self, filename):
        hidden_text_content = self.read_hidden_text(filename)
        # Convert each byte to its binary representation and join them
        return len("".join(format(byte, "08b") for byte in hidden_text_content))

    def find_raw_video_file(self, filename):
        file_extensions = [".mp4", ".mkv", ".avi"]
        for ext in file_extensions:
            file_path = os.path.join(self.here, filename + ext)
            if os.path.isfile(file_path):
                return file_path
        return None

    def convert_video(self, input_file, ffv1_video):
        # this function is the same as running this command line
        # ffmpeg -i video.mp4 -t 12 -c:v ffv1 -level 3 -coder 1 -context 1 -g 1 -slices 4 -slicecrc 1 -c:a copy output.mkv

        # in order to run any ffmpeg subprocess, you have to have ffmpeg installed into the computer.
        # https://ffmpeg.org/download.html

        # WARNING:
        # the ffmpeg you should download is not the same as the ffmpeg library for python.
        # you need to download the exe from the link above, then add ffmpeg bin directory to system variables
        output_file = os.path.join(self.here, ffv1_video)

        if not output_file.endswith(".mkv"):
            output_file += ".mkv"

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_file,
            "-t",
            "12",
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-c:a",
            "copy",
            output_file,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Conversion successful: {output_file}")
            return output_file
        except subprocess.CalledProcessError as e:
            print(f"Error during conversion: {e}")

    def extract_audio(self, ffv1_video, audio_path):
        # Ensure the audio output file has the correct extension
        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        # Full path to the extracted audio file
        extracted_audio = os.path.join(self.here, audio_path)

        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        command = [
            "ffmpeg",
            "-i",
            ffv1_video,
            "-q:a",
            "0",
            "-map",
            "a",
            extracted_audio,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            print(f"Audio successfully extracted to {extracted_audio}")
            print(result.stdout.decode())
            print(result.stderr.decode())
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            print(e.stdout.decode())
            print(e.stderr.decode())

    def read_frame_binary(self, frame_path):
        # Open the image and convert to binary
        with open(frame_path, "rb") as f:
            binary_content = f.read()
            binary_string = "".join(format(byte, "08b") for byte in binary_content)
        return binary_string

    def remove_frames_from_video(self, input_video, frames_to_remove):
        if not input_video.endswith(".mkv"):
            input_video += ".mkv"

        input_video_path = os.path.join(self.here, input_video)

        # Create a filter string to exclude specific frames
        filter_str = (
            "select='not("
            + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
            + ")',setpts=N/FRAME_RATE/TB"
        )

        # Temporary output video path
        output_video_path = os.path.join(self.here, "temp_output.mkv")

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_video_path,
            "-vf",
            filter_str,
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-an",  # Remove audio
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Frames removed. Temporary video created at {output_video_path}")

            # Replace the original video with the new video
            os.replace(output_video_path, input_video_path)
            print(f"Original video replaced with updated video at {input_video_path}")

            # Re-add the trimmed audio to the new video
            self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            if os.path.exists(output_video_path):
                os.remove(output_video_path)

    def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
        # Calculate the new duration based on the remaining frames
        fps = 60  # Assuming the framerate is 60 fps
        total_frames_removed = len(frames_to_remove)
        original_duration = self.get_video_duration(video_path)
        new_duration = original_duration - (total_frames_removed / fps)

        # Extract and trim the audio
        audio_path = os.path.join(self.here, "trimmed_audio.aac")
        command_extract_trim = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-t",
            str(new_duration),
            "-q:a",
            "0",
            "-map",
            "a",
            audio_path,
        ]
        try:
            subprocess.run(command_extract_trim, check=True)
            print(f"Audio successfully trimmed and extracted to {audio_path}")

            # Add the trimmed audio back to the video
            final_video_path = video_path.replace(".mkv", "_final.mkv")
            command_add_audio = [
                "ffmpeg",
                "-y",
                "-i",
                video_path,
                "-i",
                audio_path,
                "-c:v",
                "copy",
                "-c:a",
                "aac",
                "-strict",
                "experimental",
                final_video_path,
            ]
            subprocess.run(command_add_audio, check=True)
            print(f"Final video with trimmed audio created at {final_video_path}")

            # Replace the original video with the final video
            os.replace(final_video_path, video_path)
            print(f"Original video replaced with final video at {video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def get_video_duration(self, video_path):
        command = [
            "ffprobe",
            "-v",
            "error",
            "-show_entries",
            "format=duration",
            "-of",
            "default=noprint_wrappers=1:nokey=1",
            video_path,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            duration = float(result.stdout.decode().strip())
            return duration
        except subprocess.CalledProcessError as e:
            print(f"An error occurred while getting video duration: {e}")
            return 0.0

    def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)

    def stitch_frames_to_video(self, ffv1_video, framerate=60):
        # this command is another ffmpeg subcommand.
        # it takes every single frame from data1 directory and stitch it back into a ffv1 video
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        output_video_path = os.path.join(self.here, ffv1_video)

        command = [
            "ffmpeg",
            "-y",
            "-framerate",
            str(framerate),
            "-i",
            os.path.join(self.frames_directory, "frame%d.png"),
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Video successfully created at {output_video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def add_audio_to_video(self, encoded_video, audio_path, final_video):
        # the audio will be lost during splitting and restitching.
        # that is why previously we separated the audio from video and saved it as aac.
        # now, we can put the audio back into the video, again using ffmpeg subcommand.

        if not encoded_video.endswith(".mkv"):
            encoded_video += ".mkv"

        if not final_video.endswith(".mkv"):
            final_video += ".mkv"

        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        final_output_path = os.path.join(self.here, final_video)

        command = [
            "ffmpeg",
            "-y",
            "-i",
            os.path.join(self.here, encoded_video),
            "-i",
            os.path.join(self.here, audio_path),
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_output_path,
        ]
        try:
            subprocess.run(command, check=True)
            print(f"Final video with audio created at {final_output_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def concatenate_videos(self, video1_path, video2_path, output_path):
        if not video1_path.endswith(".mkv"):
            video1_path += ".mkv"
        if not video2_path.endswith(".mkv"):
            video2_path += ".mkv"
        if not output_path.endswith(".mkv"):
            output_path += ".mkv"

        video1_path = os.path.join(self.here, video1_path)
        video2_path = os.path.join(self.here, video2_path)
        output_video_path = os.path.join(self.here, output_path)

        # Create a text file with the paths of the videos to concatenate
        concat_list_path = os.path.join(self.here, "concat_list.txt")
        with open(concat_list_path, "w") as f:
            f.write(f"file '{video1_path}'\n")
            f.write(f"file '{video2_path}'\n")

        command = [
            "ffmpeg",
            "-y",
            "-f",
            "concat",
            "-safe",
            "0",
            "-i",
            concat_list_path,
            "-c",
            "copy",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Videos successfully concatenated into {output_video_path}")
            os.remove(concat_list_path)  # Clean up the temporary file
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def cleanup(self, files_to_delete):
        # Delete specified files
        for file in files_to_delete:
            file_path = os.path.join(self.here, file)
            if os.path.exists(file_path):
                os.remove(file_path)
                print(f"Deleted file: {file_path}")
            else:
                print(f"File not found: {file_path}")

        # Delete the frames directory and its contents
        if os.path.exists(self.frames_directory):
            shutil.rmtree(self.frames_directory)
            print(f"Deleted directory and its contents: {self.frames_directory}")
        else:
            print(f"Directory not found: {self.frames_directory}")


if __name__ == "__main__":
    stego = FFV1Steganography()

    # original video (mp4,mkv,avi)
    original_video = "video"
    # converted ffv1 video
    ffv1_video = "output"
    # extracted audio
    extracted_audio = "audio"
    # encoded video without sound
    encoded_video = "encoded"
    # final result video, encoded, with sound
    final_video = "result"

    # region --hidden text processing --
    hidden_text = stego.read_hidden_text("hiddentext.txt")
    hidden_text_length = stego.calculate_length_of_hidden_text("hiddentext.txt")
    # endregion

    # region -- raw video locating --
    raw_video_file = stego.find_raw_video_file(original_video)
    if raw_video_file:
        print(f"Found video file: {raw_video_file}")
    else:
        print("video.mp4 not found.")
    # endregion

    # region -- video processing INPUT--
    # converted_video_file = stego.convert_video(raw_video_file, ffv1_video)
    # if converted_video_file and os.path.exists(converted_video_file):
    #     stego.extract_audio(converted_video_file, extracted_audio)
    # else:
    #     print(f"Conversion failed: {converted_video_file} not found.")

    # stego.split_into_frames(ffv1_video, hidden_text_length * 50000)
    # endregion

    # region -- video processing RESULT --
    # stego.stitch_frames_to_video(encoded_video)
    stego.concatenate_videos(encoded_video, ffv1_video, final_video)
    # stego.add_audio_to_video(final_video, extracted_audio, final_video)
    # endregion

    # region -- cleanup --
    files_to_delete = [
        extracted_audio + ".aac",
        encoded_video + ".mkv",
        ffv1_video + ".mkv",
    ]

 stego.cleanup(files_to_delete)
    # endregion







  


  Edit for results expectations :
I dont know if there is a way to match the exact color encoding between the stitched png frames and the rest of the ffv1 video. Is there a way I can extract the frames such that the color, encoding or anything I may not know about ffv1 match the original ffv1 video ?