Recherche avancée

Sur d’autres sites (8025)

• Announcing our latest open source project : DeviceDetector

  30 juillet 2014, par Stefan Giehl — Community, Development, Meta, DeviceDetector

  This blog post is an announcement for our latest open source project release : DeviceDetector ! The Universal Device Detection library will parse any User Agent and detect the browser, operating system, device used (desktop, tablet, mobile, tv, cars, console, etc.), brand and model.

  Read on to learn more about this exciting release.

  Why did we create DeviceDetector ?

  Our previous library UserAgentParser only had the possibility to detect operating systems and browsers. But as more and more traffic is coming from mobile devices like smartphones and tablets it is getting more and more important to know which devices are used by the websites visitors.

  To ensure that the device detection within Piwik will gain the required attention, so it will be as accurate as possible, we decided to move that part of Piwik into a separate project, that we will maintain separately. As an own project we hope the DeviceDetector will gain a better visibility as well as a better support by and for the community !

  DeviceDetector is hosted on GitHub at piwik/device-detector. It is also available as composer package through Packagist.

  How DeviceDetector works

  Every client requesting data from a webserver identifies itself by sending a so-called User-Agent within the request to the server. Those User Agents might contain several information such as :

  • client name and version (clients can be browsers or other software like feed readers, media players, apps,…)
  • operating system name and version
  • device identifier, which can be used to detect the brand and model.

  For Example :

  Mozilla/5.0 (Linux; Android 4.4.2; Nexus 5 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.99 Mobile Safari/537.36

  This User Agent contains following information :

  Operating system is Android 4.4.2, client uses the browser Chrome Mobile 32.0.1700.99 and the device is a Google Nexus 5 smartphone.

  What DeviceDetector currently detects

  DeviceDetector is able to detect bots, like search engines, feed fetchers, site monitors and so on, five different client types, including around 100 browsers, 15 feed readers, some media players, personal information managers (like mail clients) and mobile apps using the AFNetworking framework, around 80 operating systems and nine different device types (smartphones, tablets, feature phones, consoles, tvs, car browsers, cameras, smart displays and desktop devices) from over 180 brands.

  Note : Piwik itself currently does not use the full feature set of DeviceDetector. Client detection is currently not implemented in Piwik (only detected browsers are reported, other clients are marked as Unknown). Client detection will be implemented into Piwik in the future, follow #5413 to stay updated.

  Performance of DeviceDetector

  Our detections are currently handled by an enormous number of regexes, that are defined in several .YML Files. As parsing these .YML files is a bit slow, DeviceDetector is able to cache the parsed .YML Files. By default DeviceDetector uses a static cache, which means that everything is cached in static variables. As that only improves speed for many detections within one process, there are also adapters to cache in files or memcache for speeding up detections across requests.

  How can users help contribute to DeviceDetector ?

  Submit your devices that are not detected yet

  If you own a device, that is currently not correctly detected by the DeviceDetector, please create a issue on GitHub
  In order to check if your device is detected correctly by the DeviceDetector go to your Piwik server, click on ‘Settings’ link, then click on ‘Device Detection’ under the Diagnostic menu. If the data does not match, please copy the displayed User Agent and use that and your device data to create a ticket.

  Submit a list of your User Agents

  In order to create new detections or improve the existing ones, it is necessary for us to have lists of User Agents. If you have a website used by mostly non desktop devices it would be useful if you send a list of the User Agents that visited your website. To do so you need access to your access logs. The following command will extract the User Agents :

  zcat ~/path/to/access/logs* | awk -F'"' '{print $6}' | sort | uniq -c | sort -rn | head -n20000 > /home/piwik/top-user-agents.txt

  If you want to help us with those data, please get in touch at devicedetector@piwik.org

  Submit improvements on GitHub

  As DeviceDetector is free/libre library, we invite you to help us improving the detections as well as the code. Please feel free to create tickets and pull requests on Github.

  What’s the next big thing for DeviceDetector ?

  Please check out the list of issues in device-detector issue tracker.

  We hope the community will answer our call for help. Together, we can build DeviceDetector as the most powerful device detection library !

  Happy Device Detection,

• Frames took with ELP camera has unknown pixel format at FHD ?

  11 novembre 2024, par Marcel Kopera

  I'm trying to take a one frame ever x seconds from my usb camera. Name of the camera is : ELP-USBFHD06H-SFV(5-50).
Code is not 100% done yet, but I'm using it this way right now ↓ (shot fn is called from main.py in a loop)

  


  
import cv2
import subprocess

from time import sleep
from collections import namedtuple

from errors import *

class Camera:
    def __init__(self, cam_index, res_width, res_height, pic_format, day_time_exposure_ms, night_time_exposure_ms):
        Resolution = namedtuple("resolution", ["width", "height"])
        self.manual_mode(True)

        self.cam_index = cam_index
        self.camera_resolution = Resolution(res_width, res_height)
        self.picture_format = pic_format
        self.day_time_exposure_ms = day_time_exposure_ms
        self.night_time_exposure_ms = night_time_exposure_ms

        self.started: bool = False
        self.night_mode = False

        self.cap = cv2.VideoCapture(self.cam_index, cv2.CAP_V4L2)
        self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, self.camera_resolution.width)
        self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, self.camera_resolution.height)
        self.cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*self.picture_format))

    

    def start(self):
        sleep(1)
        if not self.cap.isOpened():
            return CameraCupError()

        self.set_exposure_time(self.day_time_exposure_ms)
        self.set_brightness(0)
        sleep(0.1)
        
        self.started = True



    def shot(self, picture_name, is_night):
        if not self.started:
            return InitializationError()

        self.configure_mode(is_night)

        # Clear buffer
        for _ in range(5):
            ret, _ = self.cap.read()

        ret, frame = self.cap.read()

        sleep(0.1)

        if ret:
            print(picture_name)
            cv2.imwrite(picture_name, frame)
            return True

        else:
            print("No photo")
            return False


    
    def release(self):
        self.set_exposure_time(156)
        self.set_brightness(0)
        self.manual_mode(False)
        self.cap.release()



    def manual_mode(self, switch: bool):
        if switch:
            subprocess.run(["v4l2-ctl", "--set-ctrl=auto_exposure=1"])
        else:
            subprocess.run(["v4l2-ctl", "--set-ctrl=auto_exposure=3"])
        sleep(1)

    
    
    def configure_mode(self, is_night):
        if is_night == self.night_mode:
            return

        if is_night:
            self.night_mode = is_night
            self.set_exposure_time(self.night_time_exposure_ms)
            self.set_brightness(64)
        else:
            self.night_mode = is_night
            self.set_exposure_time(self.day_time_exposure_ms)
            self.set_brightness(0)
        sleep(0.1)



    def set_exposure_time(self, ms: int):
        ms = int(ms)
        default_val = 156

        if ms < 1 or ms > 5000:
            ms = default_val

        self.cap.set(cv2.CAP_PROP_EXPOSURE, ms)



    def set_brightness(self, value: int):
        value = int(value)
        default_val = 0

        if value < -64 or value > 64:
            value = default_val

        self.cap.set(cv2.CAP_PROP_BRIGHTNESS, value)


  


  Here are settings for the camera (yaml file)

  


  camera:
  camera_index: 0
  res_width: 1920
  res_height: 1080
  picture_format: "MJPG"
  day_time_exposure_ms: 5
  night_time_exposure_ms: 5000
  photos_format: "jpg"



  


  I do some configs like set manual mode for the camera, change exposure/brightness and saving frame.
Also the camera is probably catching the frames to the buffer (it is not saving latest frame in real time : it's more laggish), so I have to clear buffer every time. like this

  


          # Clear buffer from old frames
        for _ in range(5):
            ret, _ = self.cap.read()
        
        # Get a new frame
        ret, frame = self.cap.read()


  


  What I really don't like, but I could find a better way (tldr : setting buffer for 1 frame doesn't work on my camera).

  


  Frames saved this method looks good with 1920x1080 resolution. BUT when I try to run ffmpeg command to make a timelapse from saved jpg file like this

  


  ffmpeg -framerate 20 -pattern_type glob -i "*.jpg" -c:v libx264 output.mp4


  


  I got an error like this one

  


  [image2 @ 0x555609c45240] Could not open file : 08:59:20.jpg
[image2 @ 0x555609c45240] Could not find codec parameters for stream 0 (Video: mjpeg, none(bt470bg/unknown/unknown)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, image2, from '*.jpg':
  Duration: 00:00:00.05, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: mjpeg, none(bt470bg/unknown/unknown), 20 fps, 20 tbr, 20 tbn
Output #0, mp4, to 'output.mp4':
Output file #0 does not contain any stream


  


  Also when I try to copy the files from Linux to Windows I get some weird copy failing error and option to skip the picture. But even when I press the skip button, the picture is copied and can be opened. I'm not sure what is wrong with the format, but the camera is supporting MPEG at 1920x1080.

  


  >>> v4l2-ctl --all

Driver Info:
        Driver name      : uvcvideo
        Card type        : H264 USB Camera: USB Camera
        Bus info         : usb-xhci-hcd.1-1
        Driver version   : 6.6.51
        Capabilities     : 0x84a00001
                Video Capture
                Metadata Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : uvcvideo
        Model            : H264 USB Camera: USB Camera
        Serial           : 2020032801
        Bus info         : usb-xhci-hcd.1-1
        Media version    : 6.6.51
        Hardware revision: 0x00000100 (256)
        Driver version   : 6.6.51
Interface Info:
        ID               : 0x03000002
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : H264 USB Camera: USB Camera
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x0100000d   : 0: Sink
          Link 0x0200001a: from remote pad 0x1000010 of entity 'Extension 4' (Video Pixel Formatter): Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'MJPG' (Motion-JPEG)
        Field             : None
        Bytes per Line    : 0
        Size Image        : 4147789
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :
Crop Capability Video Capture:
        Bounds      : Left 0, Top 0, Width 1920, Height 1080
        Default     : Left 0, Top 0, Width 1920, Height 1080
        Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 1920, Height 1080, Flags:
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags:
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 15.000 (15/1)
        Read buffers     : 0

User Controls

                     brightness 0x00980900 (int)    : min=-64 max=64 step=1 default=0 value=64
                       contrast 0x00980901 (int)    : min=0 max=64 step=1 default=32 value=32
                     saturation 0x00980902 (int)    : min=0 max=128 step=1 default=56 value=56
                            hue 0x00980903 (int)    : min=-40 max=40 step=1 default=0 value=0
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1
                          gamma 0x00980910 (int)    : min=72 max=500 step=1 default=100 value=100
                           gain 0x00980913 (int)    : min=0 max=100 step=1 default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=1 value=1 (50 Hz)
                                0: Disabled
                                1: 50 Hz
                                2: 60 Hz
      white_balance_temperature 0x0098091a (int)    : min=2800 max=6500 step=1 default=4600 value=4600 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=6 step=1 default=3 value=3
         backlight_compensation 0x0098091c (int)    : min=0 max=2 step=1 default=1 value=1

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=3 value=1 (Manual Mode)
                                1: Manual Mode
                                3: Aperture Priority Mode
         exposure_time_absolute 0x009a0902 (int)    : min=1 max=5000 step=1 default=156 value=5000
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=0


  


  I also tried to save the picture using ffmpeg in a case something is not right with opencv like this :

  


  ffmpeg -f v4l2 -framerate 30 -video_size 1920x1080 -i /dev/video0 -c:v libx264 -preset fast -crf 23 -t 00:01:00 output.mp4



  


  It is saving the picture but also changing its format

  


  [video4linux2,v4l2 @ 0x555659ed92b0] The V4L2 driver changed the video from 1920x1080 to 800x600
[video4linux2,v4l2 @ 0x555659ed92b0] The driver changed the time per frame from 1/30 to 1/15


  


  But the format looks right when set it back to FHD using v4l2

  


  
>>> v4l2-ctl --device=/dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=MJPG
>>> v4l2-ctl --get-fmt-video

Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'MJPG' (Motion-JPEG)
        Field             : None
        Bytes per Line    : 0
        Size Image        : 4147789
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :


  


  I'm not sure what could be wrong with the format/camera and I don't think I have enough information to figure it out.

  


  I tried to use ffmpeg instead of opencv and also change a few settings in opencv's cup config.

  


• How to Stream Audio from Google Cloud Storage in Chunks and Convert Each Chunk to WAV for Whisper Transcription

  14 novembre 2024, par Douglas Landvik

  I'm working on a project where I need to transcribe audio stored in a Google Cloud Storage bucket using OpenAI's Whisper model. The audio is stored in WebM format with Opus encoding, and due to the file size, I'm streaming the audio in 30-second chunks.

  


  To convert each chunk to WAV (16 kHz, mono, 16-bit PCM) compatible with Whisper, I'm using FFmpeg. The first chunk converts successfully, but subsequent chunks fail to convert. I suspect this is because each chunk lacks the WebM container's header, which FFmpeg needs to interpret the Opus codec correctly.

  


  Here’s a simplified version of my approach :

  


  Download Chunk : I download each chunk from GCS as bytes.
Convert with FFmpeg : I pass the bytes to FFmpeg to convert each chunk from WebM/Opus to WAV.

  


  async def handle_transcription_and_notify(
    consultation_service: ConsultationService,
    consultation_processor: ConsultationProcessor,
    consultation: Consultation,
    language: str,
    notes: str,
    clinic_id: str,
    vet_email: str,
    trace_id: str,
    blob_path: str,
    max_retries: int = 3,
    retry_delay: int = 5,
    max_concurrent_tasks: int = 3
):
    """
    Handles the transcription process by streaming the file from GCS, converting to a compatible format, 
    and notifying the client via WebSocket.
    """
    chunk_duration_sec = 30  # 30 seconds per chunk
    logger.info(f"Starting transcription process for consultation {consultation.consultation_id}",
                extra={'trace_id': trace_id})

    # Initialize GCS client
    service_account_key = os.environ.get('SERVICE_ACCOUNT_KEY_BACKEND')
    if not service_account_key:
        logger.error("Service account key not found in environment variables", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Service account key not found for consultation {consultation.consultation_id}.\nTrace ID: {trace_id}"
        )
        return

    try:
        service_account_info = json.loads(service_account_key)
        credentials = service_account.Credentials.from_service_account_info(service_account_info)
    except Exception as e:
        logger.error(f"Error loading service account credentials: {str(e)}", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error loading service account credentials for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    # Initialize GCS client
    service_account_key = os.environ.get('SERVICE_ACCOUNT_KEY_BACKEND')
    if not service_account_key:
        logger.error("Service account key not found in environment variables", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Service account key not found for consultation {consultation.consultation_id}.\nTrace ID: {trace_id}"
        )
        return

    try:
        service_account_info = json.loads(service_account_key)
        credentials = service_account.Credentials.from_service_account_info(service_account_info)
    except Exception as e:
        logger.error(f"Error loading service account credentials: {str(e)}", extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error loading service account credentials for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    storage_client = storage.Client(credentials=credentials)
    bucket_name = 'vetz_consultations'
    blob = storage_client.bucket(bucket_name).get_blob(blob_path)
    bytes_per_second = 16000 * 2  # 32,000 bytes per second
    chunk_size_bytes = 30 * bytes_per_second
    size = blob.size

    async def stream_blob_in_chunks(blob, chunk_size):
        loop = asyncio.get_running_loop()
        start = 0
        size = blob.size
        while start < size:
            end = min(start + chunk_size - 1, size - 1)
            try:
                logger.info(f"Requesting chunk from {start} to {end}", extra={'trace_id': trace_id})
                chunk = await loop.run_in_executor(
                    None, lambda: blob.download_as_bytes(start=start, end=end)
                )
                if not chunk:
                    break
                logger.info(f"Yielding chunk from {start} to {end}, size: {len(chunk)} bytes",
                            extra={'trace_id': trace_id})
                yield chunk
                start += chunk_size
            except Exception as e:
                logger.error(f"Error downloading chunk from {start} to {end}: {str(e)}", exc_info=True,
                             extra={'trace_id': trace_id})
                raise e

    async def convert_to_wav(chunk_bytes, chunk_idx):
        """
        Convert audio chunk to WAV format compatible with Whisper, ensuring it's 16 kHz, mono, and 16-bit PCM.
        """
        try:
            logger.debug(f"Processing chunk {chunk_idx}: size = {len(chunk_bytes)} bytes")

            detected_format = await detect_audio_format(chunk_bytes)
            logger.info(f"Detected audio format for chunk {chunk_idx}: {detected_format}")
            input_io = io.BytesIO(chunk_bytes)
            output_io = io.BytesIO()

            # ffmpeg command to convert webm/opus to WAV with 16 kHz, mono, and 16-bit PCM

            # ffmpeg command with debug information
            ffmpeg_command = [
                "ffmpeg",
                "-loglevel", "debug",
                "-f", "s16le",            # Treat input as raw PCM data
                "-ar", "48000",           # Set input sample rate
                "-ac", "1",               # Set input to mono
                "-i", "pipe:0",
                "-ar", "16000",           # Set output sample rate to 16 kHz
                "-ac", "1",               # Ensure mono output
                "-sample_fmt", "s16",     # Set output format to 16-bit PCM
                "-f", "wav",              # Output as WAV format
                "pipe:1"
            ]

            process = subprocess.Popen(
                ffmpeg_command,
                stdin=subprocess.PIPE,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE
            )

            stdout, stderr = process.communicate(input=input_io.read())

            if process.returncode == 0:
                logger.info(f"FFmpeg conversion completed successfully for chunk {chunk_idx}")
                output_io.write(stdout)
                output_io.seek(0)

                # Save the WAV file locally for listening
                output_dir = "converted_chunks"
                os.makedirs(output_dir, exist_ok=True)
                file_path = os.path.join(output_dir, f"chunk_{chunk_idx}.wav")

                with open(file_path, "wb") as f:
                    f.write(stdout)
                logger.info(f"Chunk {chunk_idx} saved to {file_path}")

                return output_io
            else:
                logger.error(f"FFmpeg failed for chunk {chunk_idx} with return code {process.returncode}")
                logger.error(f"Chunk {chunk_idx} - FFmpeg stderr: {stderr.decode()}")
                return None

        except Exception as e:
            logger.error(f"Unexpected error in FFmpeg conversion for chunk {chunk_idx}: {str(e)}")
            return None

    async def transcribe_chunk(idx, chunk_bytes):
        for attempt in range(1, max_retries + 1):
            try:
                logger.info(f"Transcribing chunk {idx + 1} (attempt {attempt}).", extra={'trace_id': trace_id})

                # Convert to WAV format
                wav_io = await convert_to_wav(chunk_bytes, idx)
                if not wav_io:
                    logger.error(f"Failed to convert chunk {idx + 1} to WAV format.")
                    return ""

                wav_io.name = "chunk.wav"
                chunk_transcription = await consultation_processor.transcribe_audio_whisper(wav_io)
                logger.info(f"Chunk {idx + 1} transcribed successfully.", extra={'trace_id': trace_id})
                return chunk_transcription
            except Exception as e:
                logger.error(f"Error transcribing chunk {idx + 1} (attempt {attempt}): {str(e)}", exc_info=True,
                             extra={'trace_id': trace_id})
                if attempt < max_retries:
                    await asyncio.sleep(retry_delay)
                else:
                    await send_discord_alert(
                        f"Max retries reached for chunk {idx + 1} in consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
                    )
                    return ""  # Return empty string for failed chunk

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} is being transcribed.", vet_email
    )

    try:
        idx = 0
        full_transcription = []
        async for chunk in stream_blob_in_chunks(blob, chunk_size_bytes):
            transcription = await transcribe_chunk(idx, chunk)
            if transcription:
                full_transcription.append(transcription)
            idx += 1

        combined_transcription = " ".join(full_transcription)
        consultation.full_transcript = (consultation.full_transcript or "") + " " + combined_transcription
        consultation_service.save_consultation(clinic_id, vet_email, consultation)
        logger.info(f"Transcription saved for consultation {consultation.consultation_id}.",
                    extra={'trace_id': trace_id})

    except Exception as e:
        logger.error(f"Error during transcription process: {str(e)}", exc_info=True, extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error during transcription process for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        return

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} has been transcribed.", vet_email
    )

    try:
        template_service = TemplateService()
        medical_record_template = template_service.get_template_by_name(
            consultation.medical_record_template_id).sections

        sections = await consultation_processor.extract_structured_sections(
            transcription=consultation.full_transcript,
            notes=notes,
            language=language,
            template=medical_record_template,
        )
        consultation.sections = sections
        consultation_service.save_consultation(clinic_id, vet_email, consultation)
        logger.info(f"Sections processed for consultation {consultation.consultation_id}.",
                    extra={'trace_id': trace_id})
    except Exception as e:
        logger.error(f"Error processing sections for consultation {consultation.consultation_id}: {str(e)}",
                     exc_info=True, extra={'trace_id': trace_id})
        await send_discord_alert(
            f"Error processing sections for consultation {consultation.consultation_id}.\nError: {str(e)}\nTrace ID: {trace_id}"
        )
        raise e

    await notification_manager.send_personal_message(
        f"Consultation {consultation.consultation_id} is fully processed.", vet_email
    )
    logger.info(f"Successfully processed consultation {consultation.consultation_id}.",
                extra={'trace_id': trace_id})