Recherche avancée

Sur d’autres sites (8310)

• My stitched frames colors looks very different from my original video, causing my video to not be able to stitch it back properly [closed]

  27 mai 2024, par Wer Wer

  I am trying to extract some frames off my video to do some form of steganography. I accidentally used a 120fps video, causing the files to be too big when i extract every single frame. To fix this, I decided to calculate how many frames is needed to hide the bits (LSB replacement for every 8 bit) and then extract only certain amount of frames. This means

  


  

  1. if i only need 1 frame, ill extract frame0.png

  


  2. ill remove frame0 from the original video

  


  3. encode my data into frame0.png

  


  4. stitch frame0 back into ffv1 video

  


  5. concatenate frame0 video to the rest of the video, frame0 video in front.

  


  


  I can do extraction and remove frame0 from the video. However, when looking at frame0.mkv and the original.mkv, i realised the colors seemed to be different.
Frame0.mkv
original.mkv

  


  This causes a glitch during the stitching of videos together, where the end of the video has some corrupted pixels. Not only that, it stops the video at where frame0 ends. I think those corrupted pixels were supposed to be original.mkv pixels, but they did not concatenate properly.
results.mkv

  


  I use an ffmpeg sub command to extract frames and stitch them

  


      def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)



  


  This code splits the video into the required number of frames. It checks if the total amount of frame bits is enough to encode the hidden text

  


  def remove_frames_from_video(self, input_video, frames_to_remove):
    if not input_video.endswith(".mkv"):
        input_video += ".mkv"

    input_video_path = os.path.join(self.here, input_video)

    # Create a filter string to exclude specific frames
    filter_str = (
        "select='not("
        + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
        + ")',setpts=N/FRAME_RATE/TB"
    )

    # Temporary output video path
    output_video_path = os.path.join(self.here, "temp_output.mkv")

    command = [
        "ffmpeg",
        "-y",
        "-i",
        input_video_path,
        "-vf",
        filter_str,
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        "-an",  # Remove audio
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Frames removed. Temporary video created at {output_video_path}")

        # Replace the original video with the new video
        os.replace(output_video_path, input_video_path)
        print(f"Original video replaced with updated video at {input_video_path}")

        # Re-add the trimmed audio to the new video
        self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")
        if os.path.exists(output_video_path):
            os.remove(output_video_path)

def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
    # Calculate the new duration based on the remaining frames
    fps = 60  # Assuming the framerate is 60 fps
    total_frames_removed = len(frames_to_remove)
    original_duration = self.get_video_duration(video_path)
    new_duration = original_duration - (total_frames_removed / fps)

    # Extract and trim the audio
    audio_path = os.path.join(self.here, "trimmed_audio.aac")
    command_extract_trim = [
        "ffmpeg",
        "-y",
        "-i",
        video_path,
        "-t",
        str(new_duration),
        "-q:a",
        "0",
        "-map",
        "a",
        audio_path,
    ]
    try:
        subprocess.run(command_extract_trim, check=True)
        print(f"Audio successfully trimmed and extracted to {audio_path}")

        # Add the trimmed audio back to the video
        final_video_path = video_path.replace(".mkv", "_final.mkv")
        command_add_audio = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-i",
            audio_path,
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_video_path,
        ]
        subprocess.run(command_add_audio, check=True)
        print(f"Final video with trimmed audio created at {final_video_path}")

        # Replace the original video with the final video
        os.replace(final_video_path, video_path)
        print(f"Original video replaced with final video at {video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")

def get_video_duration(self, video_path):
    command = [
        "ffprobe",
        "-v",
        "error",
        "-show_entries",
        "format=duration",
        "-of",
        "default=noprint_wrappers=1:nokey=1",
        video_path,
    ]
    try:
        result = subprocess.run(
            command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
        )
        duration = float(result.stdout.decode().strip())
        return duration
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while getting video duration: {e}")
        return 0.0


  


  here ill remove all the frames that has been extracted from the video

  


  def stitch_frames_to_video(self, ffv1_video, framerate=60):
    # this command is another ffmpeg subcommand.
    # it takes every single frame from data1 directory and stitch it back into a ffv1 video
    if not ffv1_video.endswith(".mkv"):
        ffv1_video += ".mkv"

    output_video_path = os.path.join(self.here, ffv1_video)

    command = [
        "ffmpeg",
        "-y",
        "-framerate",
        str(framerate),
        "-i",
        os.path.join(self.frames_directory, "frame%d.png"),
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Video successfully created at {output_video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  after encoding the frames, ill try to stitch the frames back into ffv1 video

  


  def concatenate_videos(self, video1_path, video2_path, output_path):
    if not video1_path.endswith(".mkv"):
        video1_path += ".mkv"
    if not video2_path.endswith(".mkv"):
        video2_path += ".mkv"
    if not output_path.endswith(".mkv"):
        output_path += ".mkv"

    video1_path = os.path.join(self.here, video1_path)
    video2_path = os.path.join(self.here, video2_path)
    output_video_path = os.path.join(self.here, output_path)

    # Create a text file with the paths of the videos to concatenate
    concat_list_path = os.path.join(self.here, "concat_list.txt")
    with open(concat_list_path, "w") as f:
        f.write(f"file '{video1_path}'\n")
        f.write(f"file '{video2_path}'\n")

    command = [
        "ffmpeg",
        "-y",
        "-f",
        "concat",
        "-safe",
        "0",
        "-i",
        concat_list_path,
        "-c",
        "copy",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Videos successfully concatenated into {output_video_path}")
        os.remove(concat_list_path)  # Clean up the temporary file
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  now i try to concatenate the frames video with the original video, but it is corrupting as the colors are different.

  


  this code does the other processing by removing all the extracted frames from the video, as well as trimming the audio (but i think ill be removing the audio trimming as i realised it is not needed at all)

  


  I think its because .png frames will lose colors when they get extracted out. The only work around I know is to extract every single frame. But this causes the program to run too long as for a 12 second video, I will extract 700++ frames. Is there a way to fix this ?

  


  my full code

  


  import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


class FFV1Steganography:
    def __init__(self):
        self.here = os.path.dirname(os.path.abspath(__file__))

        # Create a folder to save the frames
        self.frames_directory = os.path.join(self.here, "data")
        try:
            if not os.path.exists(self.frames_directory):
                os.makedirs(self.frames_directory)
        except OSError:
            print("Error: Creating directory of data")

    def read_hidden_text(self, filename):
        file_path_txt = os.path.join(self.here, filename)
        # Read the content of the file in binary mode
        with open(file_path_txt, "rb") as f:
            hidden_text_content = f.read()
        return hidden_text_content

    def calculate_length_of_hidden_text(self, filename):
        hidden_text_content = self.read_hidden_text(filename)
        # Convert each byte to its binary representation and join them
        return len("".join(format(byte, "08b") for byte in hidden_text_content))

    def find_raw_video_file(self, filename):
        file_extensions = [".mp4", ".mkv", ".avi"]
        for ext in file_extensions:
            file_path = os.path.join(self.here, filename + ext)
            if os.path.isfile(file_path):
                return file_path
        return None

    def convert_video(self, input_file, ffv1_video):
        # this function is the same as running this command line
        # ffmpeg -i video.mp4 -t 12 -c:v ffv1 -level 3 -coder 1 -context 1 -g 1 -slices 4 -slicecrc 1 -c:a copy output.mkv

        # in order to run any ffmpeg subprocess, you have to have ffmpeg installed into the computer.
        # https://ffmpeg.org/download.html

        # WARNING:
        # the ffmpeg you should download is not the same as the ffmpeg library for python.
        # you need to download the exe from the link above, then add ffmpeg bin directory to system variables
        output_file = os.path.join(self.here, ffv1_video)

        if not output_file.endswith(".mkv"):
            output_file += ".mkv"

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_file,
            "-t",
            "12",
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-c:a",
            "copy",
            output_file,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Conversion successful: {output_file}")
            return output_file
        except subprocess.CalledProcessError as e:
            print(f"Error during conversion: {e}")

    def extract_audio(self, ffv1_video, audio_path):
        # Ensure the audio output file has the correct extension
        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        # Full path to the extracted audio file
        extracted_audio = os.path.join(self.here, audio_path)

        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        command = [
            "ffmpeg",
            "-i",
            ffv1_video,
            "-q:a",
            "0",
            "-map",
            "a",
            extracted_audio,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            print(f"Audio successfully extracted to {extracted_audio}")
            print(result.stdout.decode())
            print(result.stderr.decode())
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            print(e.stdout.decode())
            print(e.stderr.decode())

    def read_frame_binary(self, frame_path):
        # Open the image and convert to binary
        with open(frame_path, "rb") as f:
            binary_content = f.read()
            binary_string = "".join(format(byte, "08b") for byte in binary_content)
        return binary_string

    def remove_frames_from_video(self, input_video, frames_to_remove):
        if not input_video.endswith(".mkv"):
            input_video += ".mkv"

        input_video_path = os.path.join(self.here, input_video)

        # Create a filter string to exclude specific frames
        filter_str = (
            "select='not("
            + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
            + ")',setpts=N/FRAME_RATE/TB"
        )

        # Temporary output video path
        output_video_path = os.path.join(self.here, "temp_output.mkv")

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_video_path,
            "-vf",
            filter_str,
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-an",  # Remove audio
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Frames removed. Temporary video created at {output_video_path}")

            # Replace the original video with the new video
            os.replace(output_video_path, input_video_path)
            print(f"Original video replaced with updated video at {input_video_path}")

            # Re-add the trimmed audio to the new video
            self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            if os.path.exists(output_video_path):
                os.remove(output_video_path)

    def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
        # Calculate the new duration based on the remaining frames
        fps = 60  # Assuming the framerate is 60 fps
        total_frames_removed = len(frames_to_remove)
        original_duration = self.get_video_duration(video_path)
        new_duration = original_duration - (total_frames_removed / fps)

        # Extract and trim the audio
        audio_path = os.path.join(self.here, "trimmed_audio.aac")
        command_extract_trim = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-t",
            str(new_duration),
            "-q:a",
            "0",
            "-map",
            "a",
            audio_path,
        ]
        try:
            subprocess.run(command_extract_trim, check=True)
            print(f"Audio successfully trimmed and extracted to {audio_path}")

            # Add the trimmed audio back to the video
            final_video_path = video_path.replace(".mkv", "_final.mkv")
            command_add_audio = [
                "ffmpeg",
                "-y",
                "-i",
                video_path,
                "-i",
                audio_path,
                "-c:v",
                "copy",
                "-c:a",
                "aac",
                "-strict",
                "experimental",
                final_video_path,
            ]
            subprocess.run(command_add_audio, check=True)
            print(f"Final video with trimmed audio created at {final_video_path}")

            # Replace the original video with the final video
            os.replace(final_video_path, video_path)
            print(f"Original video replaced with final video at {video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def get_video_duration(self, video_path):
        command = [
            "ffprobe",
            "-v",
            "error",
            "-show_entries",
            "format=duration",
            "-of",
            "default=noprint_wrappers=1:nokey=1",
            video_path,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            duration = float(result.stdout.decode().strip())
            return duration
        except subprocess.CalledProcessError as e:
            print(f"An error occurred while getting video duration: {e}")
            return 0.0

    def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)

    def stitch_frames_to_video(self, ffv1_video, framerate=60):
        # this command is another ffmpeg subcommand.
        # it takes every single frame from data1 directory and stitch it back into a ffv1 video
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        output_video_path = os.path.join(self.here, ffv1_video)

        command = [
            "ffmpeg",
            "-y",
            "-framerate",
            str(framerate),
            "-i",
            os.path.join(self.frames_directory, "frame%d.png"),
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Video successfully created at {output_video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def add_audio_to_video(self, encoded_video, audio_path, final_video):
        # the audio will be lost during splitting and restitching.
        # that is why previously we separated the audio from video and saved it as aac.
        # now, we can put the audio back into the video, again using ffmpeg subcommand.

        if not encoded_video.endswith(".mkv"):
            encoded_video += ".mkv"

        if not final_video.endswith(".mkv"):
            final_video += ".mkv"

        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        final_output_path = os.path.join(self.here, final_video)

        command = [
            "ffmpeg",
            "-y",
            "-i",
            os.path.join(self.here, encoded_video),
            "-i",
            os.path.join(self.here, audio_path),
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_output_path,
        ]
        try:
            subprocess.run(command, check=True)
            print(f"Final video with audio created at {final_output_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def concatenate_videos(self, video1_path, video2_path, output_path):
        if not video1_path.endswith(".mkv"):
            video1_path += ".mkv"
        if not video2_path.endswith(".mkv"):
            video2_path += ".mkv"
        if not output_path.endswith(".mkv"):
            output_path += ".mkv"

        video1_path = os.path.join(self.here, video1_path)
        video2_path = os.path.join(self.here, video2_path)
        output_video_path = os.path.join(self.here, output_path)

        # Create a text file with the paths of the videos to concatenate
        concat_list_path = os.path.join(self.here, "concat_list.txt")
        with open(concat_list_path, "w") as f:
            f.write(f"file '{video1_path}'\n")
            f.write(f"file '{video2_path}'\n")

        command = [
            "ffmpeg",
            "-y",
            "-f",
            "concat",
            "-safe",
            "0",
            "-i",
            concat_list_path,
            "-c",
            "copy",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Videos successfully concatenated into {output_video_path}")
            os.remove(concat_list_path)  # Clean up the temporary file
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def cleanup(self, files_to_delete):
        # Delete specified files
        for file in files_to_delete:
            file_path = os.path.join(self.here, file)
            if os.path.exists(file_path):
                os.remove(file_path)
                print(f"Deleted file: {file_path}")
            else:
                print(f"File not found: {file_path}")

        # Delete the frames directory and its contents
        if os.path.exists(self.frames_directory):
            shutil.rmtree(self.frames_directory)
            print(f"Deleted directory and its contents: {self.frames_directory}")
        else:
            print(f"Directory not found: {self.frames_directory}")


if __name__ == "__main__":
    stego = FFV1Steganography()

    # original video (mp4,mkv,avi)
    original_video = "video"
    # converted ffv1 video
    ffv1_video = "output"
    # extracted audio
    extracted_audio = "audio"
    # encoded video without sound
    encoded_video = "encoded"
    # final result video, encoded, with sound
    final_video = "result"

    # region --hidden text processing --
    hidden_text = stego.read_hidden_text("hiddentext.txt")
    hidden_text_length = stego.calculate_length_of_hidden_text("hiddentext.txt")
    # endregion

    # region -- raw video locating --
    raw_video_file = stego.find_raw_video_file(original_video)
    if raw_video_file:
        print(f"Found video file: {raw_video_file}")
    else:
        print("video.mp4 not found.")
    # endregion

    # region -- video processing INPUT--
    # converted_video_file = stego.convert_video(raw_video_file, ffv1_video)
    # if converted_video_file and os.path.exists(converted_video_file):
    #     stego.extract_audio(converted_video_file, extracted_audio)
    # else:
    #     print(f"Conversion failed: {converted_video_file} not found.")

    # stego.split_into_frames(ffv1_video, hidden_text_length * 50000)
    # endregion

    # region -- video processing RESULT --
    # stego.stitch_frames_to_video(encoded_video)
    stego.concatenate_videos(encoded_video, ffv1_video, final_video)
    # stego.add_audio_to_video(final_video, extracted_audio, final_video)
    # endregion

    # region -- cleanup --
    files_to_delete = [
        extracted_audio + ".aac",
        encoded_video + ".mkv",
        ffv1_video + ".mkv",
    ]

 stego.cleanup(files_to_delete)
    # endregion







  


  Edit for results expectations :
I dont know if there is a way to match the exact color encoding between the stitched png frames and the rest of the ffv1 video. Is there a way I can extract the frames such that the color, encoding or anything I may not know about ffv1 match the original ffv1 video ?

  


• FFMPEG My stitched frames colors looks very different from my original video, causing my video to not be able to stitch it back properly

  26 mai 2024, par Wer Wer

  I am trying to extract some frames off my video to do some form of steganography. I accidentally used a 120fps video, causing the files to be too big when i extract every single frame. To fix this, I decided to calculate how many frames is needed to hide the bits (LSB replacement for every 8 bit) and then extract only certain amount of frames. This means

  


  

  1. if i only need 1 frame, ill extract frame0.png

  


  2. ill remove frame0 from the original video

  


  3. encode my data into frame0.png

  


  4. stitch frame0 back into ffv1 video

  


  5. concatenate frame0 video to the rest of the video, frame0 video in front.

  


  


  I can do extraction and remove frame0 from the video. However, when looking at frame0.mkv and the original.mkv, i realised the colors seemed to be different.
Frame0.mkv
original.mkv

  


  This causes a glitch during the stitching of videos together, where the end of the video has some corrupted pixels. Not only that, it stops the video at where frame0 ends. I think those corrupted pixels were supposed to be original.mkv pixels, but they did not concatenate properly.
results.mkv

  


  I use an ffmpeg sub command to extract frames and stitch them

  


      def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)



  


  This code splits the video into the required number of frames. It checks if the total amount of frame bits is enough to encode the hidden text

  


  def remove_frames_from_video(self, input_video, frames_to_remove):
    if not input_video.endswith(".mkv"):
        input_video += ".mkv"

    input_video_path = os.path.join(self.here, input_video)

    # Create a filter string to exclude specific frames
    filter_str = (
        "select='not("
        + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
        + ")',setpts=N/FRAME_RATE/TB"
    )

    # Temporary output video path
    output_video_path = os.path.join(self.here, "temp_output.mkv")

    command = [
        "ffmpeg",
        "-y",
        "-i",
        input_video_path,
        "-vf",
        filter_str,
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        "-an",  # Remove audio
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Frames removed. Temporary video created at {output_video_path}")

        # Replace the original video with the new video
        os.replace(output_video_path, input_video_path)
        print(f"Original video replaced with updated video at {input_video_path}")

        # Re-add the trimmed audio to the new video
        self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")
        if os.path.exists(output_video_path):
            os.remove(output_video_path)

def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
    # Calculate the new duration based on the remaining frames
    fps = 60  # Assuming the framerate is 60 fps
    total_frames_removed = len(frames_to_remove)
    original_duration = self.get_video_duration(video_path)
    new_duration = original_duration - (total_frames_removed / fps)

    # Extract and trim the audio
    audio_path = os.path.join(self.here, "trimmed_audio.aac")
    command_extract_trim = [
        "ffmpeg",
        "-y",
        "-i",
        video_path,
        "-t",
        str(new_duration),
        "-q:a",
        "0",
        "-map",
        "a",
        audio_path,
    ]
    try:
        subprocess.run(command_extract_trim, check=True)
        print(f"Audio successfully trimmed and extracted to {audio_path}")

        # Add the trimmed audio back to the video
        final_video_path = video_path.replace(".mkv", "_final.mkv")
        command_add_audio = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-i",
            audio_path,
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_video_path,
        ]
        subprocess.run(command_add_audio, check=True)
        print(f"Final video with trimmed audio created at {final_video_path}")

        # Replace the original video with the final video
        os.replace(final_video_path, video_path)
        print(f"Original video replaced with final video at {video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")

def get_video_duration(self, video_path):
    command = [
        "ffprobe",
        "-v",
        "error",
        "-show_entries",
        "format=duration",
        "-of",
        "default=noprint_wrappers=1:nokey=1",
        video_path,
    ]
    try:
        result = subprocess.run(
            command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
        )
        duration = float(result.stdout.decode().strip())
        return duration
    except subprocess.CalledProcessError as e:
        print(f"An error occurred while getting video duration: {e}")
        return 0.0


  


  here ill remove all the frames that has been extracted from the video

  


  def stitch_frames_to_video(self, ffv1_video, framerate=60):
    # this command is another ffmpeg subcommand.
    # it takes every single frame from data1 directory and stitch it back into a ffv1 video
    if not ffv1_video.endswith(".mkv"):
        ffv1_video += ".mkv"

    output_video_path = os.path.join(self.here, ffv1_video)

    command = [
        "ffmpeg",
        "-y",
        "-framerate",
        str(framerate),
        "-i",
        os.path.join(self.frames_directory, "frame%d.png"),
        "-c:v",
        "ffv1",
        "-level",
        "3",
        "-coder",
        "1",
        "-context",
        "1",
        "-g",
        "1",
        "-slices",
        "4",
        "-slicecrc",
        "1",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Video successfully created at {output_video_path}")
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  after encoding the frames, ill try to stitch the frames back into ffv1 video

  


  def concatenate_videos(self, video1_path, video2_path, output_path):
    if not video1_path.endswith(".mkv"):
        video1_path += ".mkv"
    if not video2_path.endswith(".mkv"):
        video2_path += ".mkv"
    if not output_path.endswith(".mkv"):
        output_path += ".mkv"

    video1_path = os.path.join(self.here, video1_path)
    video2_path = os.path.join(self.here, video2_path)
    output_video_path = os.path.join(self.here, output_path)

    # Create a text file with the paths of the videos to concatenate
    concat_list_path = os.path.join(self.here, "concat_list.txt")
    with open(concat_list_path, "w") as f:
        f.write(f"file '{video1_path}'\n")
        f.write(f"file '{video2_path}'\n")

    command = [
        "ffmpeg",
        "-y",
        "-f",
        "concat",
        "-safe",
        "0",
        "-i",
        concat_list_path,
        "-c",
        "copy",
        output_video_path,
    ]

    try:
        subprocess.run(command, check=True)
        print(f"Videos successfully concatenated into {output_video_path}")
        os.remove(concat_list_path)  # Clean up the temporary file
    except subprocess.CalledProcessError as e:
        print(f"An error occurred: {e}")


  


  now i try to concatenate the frames video with the original video, but it is corrupting as the colors are different.

  


  this code does the other processing by removing all the extracted frames from the video, as well as trimming the audio (but i think ill be removing the audio trimming as i realised it is not needed at all)

  


  I think its because .png frames will lose colors when they get extracted out. The only work around I know is to extract every single frame. But this causes the program to run too long as for a 12 second video, I will extract 700++ frames. Is there a way to fix this ?

  


  my full code

  


  import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


import json
import os
import shutil
import magic
import ffmpeg
import cv2
import numpy as np
import subprocess
from PIL import Image
import glob


class FFV1Steganography:
    def __init__(self):
        self.here = os.path.dirname(os.path.abspath(__file__))

        # Create a folder to save the frames
        self.frames_directory = os.path.join(self.here, "data")
        try:
            if not os.path.exists(self.frames_directory):
                os.makedirs(self.frames_directory)
        except OSError:
            print("Error: Creating directory of data")

    def read_hidden_text(self, filename):
        file_path_txt = os.path.join(self.here, filename)
        # Read the content of the file in binary mode
        with open(file_path_txt, "rb") as f:
            hidden_text_content = f.read()
        return hidden_text_content

    def calculate_length_of_hidden_text(self, filename):
        hidden_text_content = self.read_hidden_text(filename)
        # Convert each byte to its binary representation and join them
        return len("".join(format(byte, "08b") for byte in hidden_text_content))

    def find_raw_video_file(self, filename):
        file_extensions = [".mp4", ".mkv", ".avi"]
        for ext in file_extensions:
            file_path = os.path.join(self.here, filename + ext)
            if os.path.isfile(file_path):
                return file_path
        return None

    def convert_video(self, input_file, ffv1_video):
        # this function is the same as running this command line
        # ffmpeg -i video.mp4 -t 12 -c:v ffv1 -level 3 -coder 1 -context 1 -g 1 -slices 4 -slicecrc 1 -c:a copy output.mkv

        # in order to run any ffmpeg subprocess, you have to have ffmpeg installed into the computer.
        # https://ffmpeg.org/download.html

        # WARNING:
        # the ffmpeg you should download is not the same as the ffmpeg library for python.
        # you need to download the exe from the link above, then add ffmpeg bin directory to system variables
        output_file = os.path.join(self.here, ffv1_video)

        if not output_file.endswith(".mkv"):
            output_file += ".mkv"

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_file,
            "-t",
            "12",
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-c:a",
            "copy",
            output_file,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Conversion successful: {output_file}")
            return output_file
        except subprocess.CalledProcessError as e:
            print(f"Error during conversion: {e}")

    def extract_audio(self, ffv1_video, audio_path):
        # Ensure the audio output file has the correct extension
        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        # Full path to the extracted audio file
        extracted_audio = os.path.join(self.here, audio_path)

        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        command = [
            "ffmpeg",
            "-i",
            ffv1_video,
            "-q:a",
            "0",
            "-map",
            "a",
            extracted_audio,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            print(f"Audio successfully extracted to {extracted_audio}")
            print(result.stdout.decode())
            print(result.stderr.decode())
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            print(e.stdout.decode())
            print(e.stderr.decode())

    def read_frame_binary(self, frame_path):
        # Open the image and convert to binary
        with open(frame_path, "rb") as f:
            binary_content = f.read()
            binary_string = "".join(format(byte, "08b") for byte in binary_content)
        return binary_string

    def remove_frames_from_video(self, input_video, frames_to_remove):
        if not input_video.endswith(".mkv"):
            input_video += ".mkv"

        input_video_path = os.path.join(self.here, input_video)

        # Create a filter string to exclude specific frames
        filter_str = (
            "select='not("
            + "+".join([f"eq(n\,{frame})" for frame in frames_to_remove])
            + ")',setpts=N/FRAME_RATE/TB"
        )

        # Temporary output video path
        output_video_path = os.path.join(self.here, "temp_output.mkv")

        command = [
            "ffmpeg",
            "-y",
            "-i",
            input_video_path,
            "-vf",
            filter_str,
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            "-an",  # Remove audio
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Frames removed. Temporary video created at {output_video_path}")

            # Replace the original video with the new video
            os.replace(output_video_path, input_video_path)
            print(f"Original video replaced with updated video at {input_video_path}")

            # Re-add the trimmed audio to the new video
            self.trim_audio_and_add_to_video(input_video_path, frames_to_remove)
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")
            if os.path.exists(output_video_path):
                os.remove(output_video_path)

    def trim_audio_and_add_to_video(self, video_path, frames_to_remove):
        # Calculate the new duration based on the remaining frames
        fps = 60  # Assuming the framerate is 60 fps
        total_frames_removed = len(frames_to_remove)
        original_duration = self.get_video_duration(video_path)
        new_duration = original_duration - (total_frames_removed / fps)

        # Extract and trim the audio
        audio_path = os.path.join(self.here, "trimmed_audio.aac")
        command_extract_trim = [
            "ffmpeg",
            "-y",
            "-i",
            video_path,
            "-t",
            str(new_duration),
            "-q:a",
            "0",
            "-map",
            "a",
            audio_path,
        ]
        try:
            subprocess.run(command_extract_trim, check=True)
            print(f"Audio successfully trimmed and extracted to {audio_path}")

            # Add the trimmed audio back to the video
            final_video_path = video_path.replace(".mkv", "_final.mkv")
            command_add_audio = [
                "ffmpeg",
                "-y",
                "-i",
                video_path,
                "-i",
                audio_path,
                "-c:v",
                "copy",
                "-c:a",
                "aac",
                "-strict",
                "experimental",
                final_video_path,
            ]
            subprocess.run(command_add_audio, check=True)
            print(f"Final video with trimmed audio created at {final_video_path}")

            # Replace the original video with the final video
            os.replace(final_video_path, video_path)
            print(f"Original video replaced with final video at {video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def get_video_duration(self, video_path):
        command = [
            "ffprobe",
            "-v",
            "error",
            "-show_entries",
            "format=duration",
            "-of",
            "default=noprint_wrappers=1:nokey=1",
            video_path,
        ]
        try:
            result = subprocess.run(
                command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE
            )
            duration = float(result.stdout.decode().strip())
            return duration
        except subprocess.CalledProcessError as e:
            print(f"An error occurred while getting video duration: {e}")
            return 0.0

    def split_into_frames(self, ffv1_video, hidden_text_length):
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        ffv1_video_path = os.path.join(self.here, ffv1_video)
        ffv1_video = cv2.VideoCapture(ffv1_video_path)

        currentframe = 0
        total_frame_bits = 0
        frames_to_remove = []

        while True:
            ret, frame = ffv1_video.read()
            if ret:
                name = os.path.join(self.here, "data", f"frame{currentframe}.png")
                print("Creating..." + name)
                cv2.imwrite(name, frame)

                current_frame_path = os.path.join(
                    self.here, "data", f"frame{currentframe}.png"
                )

                if os.path.exists(current_frame_path):
                    binary_data = self.read_frame_binary(current_frame_path)

                if (total_frame_bits // 8) >= hidden_text_length:
                    print("Complete")
                    break
                total_frame_bits += len(binary_data)
                frames_to_remove.append(currentframe)
                currentframe += 1
            else:
                print("Complete")
                break

        ffv1_video.release()

        # Remove the extracted frames from the original video
        self.remove_frames_from_video(ffv1_video_path, frames_to_remove)

    def stitch_frames_to_video(self, ffv1_video, framerate=60):
        # this command is another ffmpeg subcommand.
        # it takes every single frame from data1 directory and stitch it back into a ffv1 video
        if not ffv1_video.endswith(".mkv"):
            ffv1_video += ".mkv"

        output_video_path = os.path.join(self.here, ffv1_video)

        command = [
            "ffmpeg",
            "-y",
            "-framerate",
            str(framerate),
            "-i",
            os.path.join(self.frames_directory, "frame%d.png"),
            "-c:v",
            "ffv1",
            "-level",
            "3",
            "-coder",
            "1",
            "-context",
            "1",
            "-g",
            "1",
            "-slices",
            "4",
            "-slicecrc",
            "1",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Video successfully created at {output_video_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def add_audio_to_video(self, encoded_video, audio_path, final_video):
        # the audio will be lost during splitting and restitching.
        # that is why previously we separated the audio from video and saved it as aac.
        # now, we can put the audio back into the video, again using ffmpeg subcommand.

        if not encoded_video.endswith(".mkv"):
            encoded_video += ".mkv"

        if not final_video.endswith(".mkv"):
            final_video += ".mkv"

        if not audio_path.endswith(".aac"):
            audio_path += ".aac"

        final_output_path = os.path.join(self.here, final_video)

        command = [
            "ffmpeg",
            "-y",
            "-i",
            os.path.join(self.here, encoded_video),
            "-i",
            os.path.join(self.here, audio_path),
            "-c:v",
            "copy",
            "-c:a",
            "aac",
            "-strict",
            "experimental",
            final_output_path,
        ]
        try:
            subprocess.run(command, check=True)
            print(f"Final video with audio created at {final_output_path}")
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def concatenate_videos(self, video1_path, video2_path, output_path):
        if not video1_path.endswith(".mkv"):
            video1_path += ".mkv"
        if not video2_path.endswith(".mkv"):
            video2_path += ".mkv"
        if not output_path.endswith(".mkv"):
            output_path += ".mkv"

        video1_path = os.path.join(self.here, video1_path)
        video2_path = os.path.join(self.here, video2_path)
        output_video_path = os.path.join(self.here, output_path)

        # Create a text file with the paths of the videos to concatenate
        concat_list_path = os.path.join(self.here, "concat_list.txt")
        with open(concat_list_path, "w") as f:
            f.write(f"file '{video1_path}'\n")
            f.write(f"file '{video2_path}'\n")

        command = [
            "ffmpeg",
            "-y",
            "-f",
            "concat",
            "-safe",
            "0",
            "-i",
            concat_list_path,
            "-c",
            "copy",
            output_video_path,
        ]

        try:
            subprocess.run(command, check=True)
            print(f"Videos successfully concatenated into {output_video_path}")
            os.remove(concat_list_path)  # Clean up the temporary file
        except subprocess.CalledProcessError as e:
            print(f"An error occurred: {e}")

    def cleanup(self, files_to_delete):
        # Delete specified files
        for file in files_to_delete:
            file_path = os.path.join(self.here, file)
            if os.path.exists(file_path):
                os.remove(file_path)
                print(f"Deleted file: {file_path}")
            else:
                print(f"File not found: {file_path}")

        # Delete the frames directory and its contents
        if os.path.exists(self.frames_directory):
            shutil.rmtree(self.frames_directory)
            print(f"Deleted directory and its contents: {self.frames_directory}")
        else:
            print(f"Directory not found: {self.frames_directory}")


if __name__ == "__main__":
    stego = FFV1Steganography()

    # original video (mp4,mkv,avi)
    original_video = "video"
    # converted ffv1 video
    ffv1_video = "output"
    # extracted audio
    extracted_audio = "audio"
    # encoded video without sound
    encoded_video = "encoded"
    # final result video, encoded, with sound
    final_video = "result"

    # region --hidden text processing --
    hidden_text = stego.read_hidden_text("hiddentext.txt")
    hidden_text_length = stego.calculate_length_of_hidden_text("hiddentext.txt")
    # endregion

    # region -- raw video locating --
    raw_video_file = stego.find_raw_video_file(original_video)
    if raw_video_file:
        print(f"Found video file: {raw_video_file}")
    else:
        print("video.mp4 not found.")
    # endregion

    # region -- video processing INPUT--
    # converted_video_file = stego.convert_video(raw_video_file, ffv1_video)
    # if converted_video_file and os.path.exists(converted_video_file):
    #     stego.extract_audio(converted_video_file, extracted_audio)
    # else:
    #     print(f"Conversion failed: {converted_video_file} not found.")

    # stego.split_into_frames(ffv1_video, hidden_text_length * 50000)
    # endregion

    # region -- video processing RESULT --
    # stego.stitch_frames_to_video(encoded_video)
    stego.concatenate_videos(encoded_video, ffv1_video, final_video)
    # stego.add_audio_to_video(final_video, extracted_audio, final_video)
    # endregion

    # region -- cleanup --
    files_to_delete = [
        extracted_audio + ".aac",
        encoded_video + ".mkv",
        ffv1_video + ".mkv",
    ]

 stego.cleanup(files_to_delete)
    # endregion







  


• Data Privacy Issues to Be Aware of and How to Overcome Them

  9 mai 2024, par Erin

  Data privacy issues are a significant concern for users globally.

  Around 76% of US consumers report that they would not buy from a company they do not trust with their data. In the European Union, a 2021 study found that around 53% of EU internet users refused to let companies access their data for advertising purposes.

  These findings send a clear message : if companies want to build consumer trust, they must honour users’ data privacy concerns. The best way to do this is by adopting transparent, ethical data collection practices — which also supports the simultaneous goal of maintaining compliance with regional data privacy acts.

  So what exactly is data privacy ?

  

  Data privacy refers to the protections that govern how personal data is collected and used, especially with respect to an individual’s control over when, where and what information they share with others.

  Data privacy also refers to the extent to which organisations and governments go to protect the personal data that they collect. Different parts of the world have different data privacy acts. These regulations outline the measures organisations must take to safeguard the data they collect from their consumers and residents. They also outline the rights of data subjects, such as the right to opt out of a data collection strategy and correct false data. 

  As more organisations rely on personal data to provide services, people have become increasingly concerned about data privacy, particularly the level of control they have over their data and what organisations and governments do with their data.

  Why should organisations take data privacy issues seriously ?

  Organisations should take data privacy seriously because consumer trust depends on it and because they have a legal obligation to do so. Doing so also helps organisations prevent threat actors from illegally accessing consumer data. Strong data privacy helps you : 

  Comply with data protection acts

  Organisations that fail to comply with regional data protection acts could face severe penalties. For example, consider the General Data Protection Regulation (GDPR), which is the primary data protection action for the European Union. The penalty system for GDPR fines consists of two tiers :

  • Less severe infringements — Which can lead to fines of up to €10 million (or 2% of an organisation’s worldwide annual revenue from the last financial year) per infringement.
  • More severe infringements — This can lead to fines of up to €20 million (or 4% of an organisation’s worldwide annual revenue from the last financial year) per infringement.

  The monetary value of these penalties is significant, so it is in the best interest of all organisations to be GDPR compliant. Other data protection acts have similar penalty systems to the GDPR. In Brazil, organisations non-compliant with the Lei Geral de Proteção de Dados Pessoais (LGPD) could be fined up to 50 million reals (USD 10 million) or 2% of their worldwide annual revenue from the last financial year.

  Improve brand reputation

  Research shows that 81% of consumers feel that how an organisation treats their data reflects how they treat them as a consumer. This means a strong correlation exists between how people perceive an organisation’s data collection practices and their other business activities.

  

  Data breaches can have a significant impact on an organisation, especially their reputation and level of consumer trust. In 2022, hackers stole customer data from the Australian private health insurance company, Medibank, and released the data onto the dark web. Optus was also affected by a cyberattack, which compromised the information of current and former customers. Following these events, a study by Nature revealed that 83 percent of Australians were concerned about the security of their data, particularly in the hands of their service providers.

  Protect consumer data

  Protecting consumer data is essential to preventing data breaches. Unfortunately, cybersecurity attacks are becoming increasingly sophisticated. In 2023 alone, organisations like T-Mobile and Sony have been compromised and their data stolen.

  One way to protect consumer data is to retain 100% data ownership. This means that no external parties can see your data. You can achieve this with the web analytics platform, Matomo. With Matomo, you can store your own data on-premises (your own servers) or in the Cloud. Under both arrangements, you retain full ownership of your data.

  Try Matomo for Free

  Get the web insights you need, while respecting user privacy.

  Start for free

  No credit card required

  What are the most pressing data privacy issues that organisations are facing today ?

  Today’s most pressing data privacy challenges organisations face are complying with new data protection acts, maintaining consumer trust, and choosing the right web analytics platform. Here is a detailed breakdown of what these challenges mean for businesses.

  Complying with new and emerging data protection laws

  Ever since the European Union introduced the GDPR in 2018, other regions have enacted similar data protection acts. In the United States, California (CCPA), Virginia (VCDPA) and Colorado have their own state-level data protection acts. Meanwhile, Brazil and China have the General Data Protection Law (LGPD) and the Personal Information Protection Law (PIPL), respectively.

  For global organisations, complying with multiple data protection acts can be tough, as each act interprets the GDPR model differently. They each have their own provisions, terminology (or different interpretations of the same terminology), and penalties.

  A web analytics platform like Matomo can help your organisation comply with the GDPR and similar data protection acts. It has a range of privacy-friendly features including data anonymisation, IP anonymisation, and first-party cookies by default. You can also create and publish custom opt-out forms and let visitors view your collected data.

  

  Today’s most pressing data privacy challenges organisations face are complying with new data protection acts, maintaining consumer trust, and choosing the right web analytics platform. Here is a detailed breakdown of what these challenges mean for businesses.

  Complying with new and emerging data protection laws

  Ever since the European Union introduced the GDPR in 2018, other regions have enacted similar data protection acts. In the United States, California (CCPA), Virginia (VCDPA) and Colorado have their own state-level data protection acts. Meanwhile, Brazil and China have the General Data Protection Law (LGPD) and the Personal Information Protection Law (PIPL), respectively.

  For global organisations, complying with multiple data protection acts can be tough, as each act interprets the GDPR model differently. They each have their own provisions, terminology (or different interpretations of the same terminology), and penalties.

  A web analytics platform like Matomo can help your organisation comply with the GDPR and similar data protection acts. It has a range of privacy-friendly features including data anonymisation, IP anonymisation, and first-party cookies by default. You can also create and publish custom opt-out forms and let visitors view your collected data.

  Try Matomo for Free

  Get the web insights you need, while respecting user privacy.

  Start for free

  No credit card required

  Maintaining consumer trust

  Building (and maintaining) consumer trust is a major hurdle for organisations. Stories about data breaches and data scandals — notably the Cambridge Analytical scandal — instil fear into the public’s hearts. After a while, people wonder, “Which company is next ?”

  One way to build and maintain trust is to be transparent about your data collection practices. Be open and honest about what data you collect (and why), where you store the data (and for how long), how you protect the data and whether you share data with third parties. 

  You should also prepare and publish your cyber incident response plan. Outline the steps you will take to contain, assess and manage a data breach.

  Choosing the right web analytics platform

  Organisations use web analytics to track and monitor web traffic, manage advertising campaigns and identify potential revenue streams. The most widely used web analytics platform is Google Analytics ; however, many users have raised concerns about privacy issues. 

  When searching for a Google Analytics alternative, consider a web analytics platform that takes data privacy seriously. Features like cookieless tracking, data anonymisation and IP anonymisation will let you track user activity without collecting personal data. Custom opt-out forms will let your web visitors enforce their data subject rights.

  What data protection acts exist right now ?

  

  As time goes on and more countries introduce their own data privacy laws, it becomes harder for organisations to adapt. Understanding the basics of each act can help streamline compliance. Here is what you need to know about the latest data protection acts.

  General Data Protection Regulation (GDPR)

  The GDPR is a data protection act created by the European Parliament and Council of the European Union. It comprises 11 chapters covering the general provisions, principles, data subject rights, penalties and other relevant information.

  The GDPR established a framework for organisations and governments to follow regarding the collection, processing, storing, transferring and deletion of personal data. Since coming into effect on 25 May 2018, other countries have used the GDPR as a model to enact similar data protection acts.

  General Data Protection Law (LGPD)

  The LGPD is Brazil’s main data protection act. The Federal Republic of Brazil signed the act on August 14, 2018, and it officially commenced on August 16, 2020. The act aimed to unify the 40 Brazilian laws that previously governed the country’s approach to processing personal data.

  Like the GDPR, the LGPD serves as a legal framework to regulate the collection and usage of personal data. It also outlines the duties of the national data protection authority, the Autoridade Nacional de Proteção de Dados (ANPD), which is responsible for enforcing the LGPD.

  Privacy Amendment (Notifiable Data Breaches) for the Privacy Act 1988

  Established by the Australian House of Representatives, the Privacy Act 1988 outlines how organisations and governments must manage personal data. The federal government has amended the Privacy Act 1988 twice — once in 2000, and again in 2014 — and is committing to a significant overhaul.

  The new proposals will make it easier for individuals to opt out of data collection, organisations will have to destroy collected data after a reasonable period, and small businesses will no longer be exempt from the Privacy Act.

  United States

  

  The United States does not have a federally mandated data protection act. Instead, each state has been gradually introducing its data protection acts, with the first being California, followed by Virginia and Colorado. Over a dozen other states are following suit, too.

  • California — The then-Governor of California Jerry Brown signed the California Consumer Privacy Act (CCPA) into law on June 28, 2018. The act applies to organisations with gross annual revenue of more than USD 25 million, and that buy or sell products and services to 100,000 or more households or consumers.
  • Virginia — The Virginia Consumer Data Protection Act (VCDPA) took effect on January 1, 2023. It applies to organisations that process (or control) the personal data of 100,000 or more consumers in a financial year. It also applies to organisations that process (or control) the personal data of 25,000 or more consumers and gain more than 50% of gross revenue by selling that data.
  • Colorado — Colorado Governor Jared Polis signed the Colorado Privacy Act (ColoPA) into law in July 2021. The act applies to organisations that process (or control) the personal data of 100,000 or more Colorado residents annually. It also applies to organisations that earn revenue from the sale of personal data of at least 25,000 Colorado residents.

  Because the US regulations are a patchwork of differing legal acts, compliance can be a complicated endeavour for organisations operating across multiple jurisdictions. 

  How can organisations comply with data protection acts ?

  One way to ensure compliance is to keep up with the latest data protection acts. But that is a very time-consuming task.

  Over 16 US states are in the process of signing new acts. And countries like China, Turkey and Australia are about to overhaul — in a big way — their own data privacy protection acts. 

  Knowledge is power. But you also have a business to run, right ? 

  That’s where Matomo comes in.

  Streamline data privacy compliance with Matomo

  Although data privacy is a major concern for individuals and companies operating in multiple parts of the world — as they must comply with new, conflicting data protection laws — it is possible to overcome the biggest data privacy issues.

  Matomo enables your visitors to take back control of their data. You can choose where you store your data on-premises and in the Cloud (EU-based). You can use various features, retain 100% data ownership, protect visitor privacy and ensure compliance.

  Try the 21-day free trial of Matomo today, start your free analytics trial. No credit card required.

  Try Matomo for Free

  21 day free trial. No credit card required.

  
  

  Your email address
  

  Your website address
  

  Your Analytics subdomain will be .matomo.cloud (change)

  Choose your analytics subdomain
  

  .matomo.cloud

  I accept the terms and conditions and data processing agreement (DPA)

  
  Your information will be used to create an account on our cloud service. For more information please consult our privacy policy.
  
  
  

  Start improving your websites now