Recherche avancée

Sur d’autres sites (10321)

• Google Speech - Streaming Request Returns EOF Error

  16 octobre 2017, par Josh

  Using Go, I’m taking a RTMP stream, transcoding it to FLAC (using ffmpeg) and attempting to stream to Google’s Speech API to transcribe the audio. However, I keep getting EOF errors when sending the data. I can’t find any information on this error in the docs so I’m not exactly sure what’s causing it.

  I’m chunking the received data into 3s clips (length isn’t relevant as long as it’s less than the maximum length of a streaming recognition request).

  Here is the core of my code :

  func main() {
  
  
  
      done := make(chan os.Signal)
  
      received := make(chan []byte)
  
  
  
      go receive(received)
  
      go transcribe(received)
  
  
  
      signal.Notify(done, os.Interrupt, syscall.SIGTERM)
  
  
  
      select {
  
      case <-done:
  
          os.Exit(0)
  
      }
  
  }
  
  
  
  func receive(received chan<- []byte) {
  
      var b bytes.Buffer
  
      stdout := bufio.NewWriter(&b)
  
  
  
      cmd := exec.Command("ffmpeg", "-i", "rtmp://127.0.0.1:1935/live/key", "-f", "flac", "-ar", "16000", "-")
  
      cmd.Stdout = stdout
  
  
  
      if err := cmd.Start(); err != nil {
  
          log.Fatal(err)
  
      }
  
  
  
      duration, _ := time.ParseDuration("3s")
  
      ticker := time.NewTicker(duration)
  
  
  
      for {
  
          select {
  
          case <-ticker.C:
  
              stdout.Flush()
  
              log.Printf("Received %d bytes", b.Len())
  
              received <- b.Bytes()
  
              b.Reset()
  
          }
  
      }
  
  }
  
  
  
  func transcribe(received <-chan []byte) {
  
      ctx := context.TODO()
  
  
  
      client, err := speech.NewClient(ctx)
  
      if err != nil {
  
          log.Fatal(err)
  
      }
  
  
  
      stream, err := client.StreamingRecognize(ctx)
  
      if err != nil {
  
          log.Fatal(err)
  
      }
  
  
  
      // Send the initial configuration message.
  
      if err = stream.Send(&speechpb.StreamingRecognizeRequest{
  
          StreamingRequest: &speechpb.StreamingRecognizeRequest_StreamingConfig{
  
              StreamingConfig: &speechpb.StreamingRecognitionConfig{
  
                  Config: &speechpb.RecognitionConfig{
  
                      Encoding:        speechpb.RecognitionConfig_FLAC,
  
                      LanguageCode:    "en-GB",
  
                      SampleRateHertz: 16000,
  
                  },
  
              },
  
          },
  
      }); err != nil {
  
          log.Fatal(err)
  
      }
  
  
  
      for {
  
          select {
  
          case data := <-received:
  
              if len(data) > 0 {
  
                  log.Printf("Sending %d bytes", len(data))
  
                  if err := stream.Send(&speechpb.StreamingRecognizeRequest{
  
                      StreamingRequest: &speechpb.StreamingRecognizeRequest_AudioContent{
  
                          AudioContent: data,
  
                      },
  
                  }); err != nil {
  
                      log.Printf("Could not send audio: %v", err)
  
                  }
  
              }
  
          }
  
      }
  
  }

  Running this code gives this output :

  2017/10/09 16:05:00 Received 191704 bytes
  
  2017/10/09 16:05:00 Saving 191704 bytes
  
  2017/10/09 16:05:00 Sending 191704 bytes
  
  2017/10/09 16:05:00 Could not send audio: EOF
  
  
  
  2017/10/09 16:05:03 Received 193192 bytes
  
  2017/10/09 16:05:03 Saving 193192 bytes
  
  2017/10/09 16:05:03 Sending 193192 bytes
  
  2017/10/09 16:05:03 Could not send audio: EOF
  
  
  
  2017/10/09 16:05:06 Received 193188 bytes
  
  2017/10/09 16:05:06 Saving 193188 bytes
  
  2017/10/09 16:05:06 Sending 193188 bytes // Notice that this doesn't error
  
  
  
  2017/10/09 16:05:09 Received 191704 bytes
  
  2017/10/09 16:05:09 Saving 191704 bytes
  
  2017/10/09 16:05:09 Sending 191704 bytes
  
  2017/10/09 16:05:09 Could not send audio: EOF

  Notice that not all of the Sends fail.

  Could anyone point me in the right direction here ? Is it something to do with the FLAC headers or something ? I also wonder if maybe resetting the buffer causes some of the data to be dropped (i.e. it’s a non-trivial operation that actually takes some time to complete) and it doesn’t like this missing information ?

  Any help would be really appreciated.

• France rules Google Analytics non-compliant with GDPR

  11 février 2022, par Erin — Privacy

  Breaking news : The French Data Protection Agency, CNIL (Commission nationale de l’informatique et des libertés), has concluded that the use of Google Analytics is illegal under GDPR. The CNIL has begun issuing formal notices to website managers using Google Analytics.

  This follows the January 2022 Austrian Data Protection Authority’s decision to declare Google Analytics illegal to use under GDPR.

  Google Analytics GDPR breaches continue to spread through the EU

  Since the invalidation of the Privacy Shield framework, an agreement between the EU and US that allowed the transfer of data to certified US companies, the CNIL and other EU data protection authorities have received numerous complaints regarding data transfers collected during visits to websites using Google Analytics.

  "It’s interesting to see that the different European Data Protection Authorities all come to the same conclusion : the use of Google Analytics is illegal. There is a European task force and we assume that this action is coordinated and other authorities will decide similarly."

  About the CNIL’s decision

  In this model case, the CNIL has found that an unnamed website’s use of Google Analytics is non-compliant with GDPR because it had breached Article 44 which prohibits the transfer of personal data beyond the EU, unless the recipient country can prove adequate data protection. 

  Under the GDPR, personal data covers a range of identifiers including email address, race, gender, phone number to name a few, but the less obvious identifiers include IP addresses or cookie IDs, for instance. 

  The CNIL’s decision was based on the fact that the US does not meet GDPR sufficient levels of data protection as a result of US surveillance laws. Therefore, the unnamed website’s use of Google Analytics created risks for their website visitors when their personal data was exported to the US. 

  At the time of writing, it is unknown if the CNIL has issued a fine for the GDPR breach. However, the website manager of the unnamed website has been ordered by the CNIL to comply with the GDPR and, if necessary, stop using Google Analytics under the current conditions.

  "One thing we’re certain of is that these decisions will continue to roll out throughout the EU and potentially beyond.

  Other countries are imposing their own privacy regulations that closely mirror the GDPR like Brazil’s General Data Protection Law (LGPD), India’s Data Protection Bill, New Zealand’s Privacy Act and Canada’s Personal Information Protection and Electronic Documents Act (PIPEDA) to name a few.”

  The CNIL offers an evaluation programme to help website managers determine whether web analytics solutions are exempt from collecting data prior to users’ agreement to opt-in through consent screens. Matomo, for instance, is a leading Google Analytics alternative that has been recommended by CNIL and is exempt from tracking consent. 

  

  English translation : “This is why I anticipated this announcement, gradually moving the analytics of my sites to @matomo_org since several weeks !

  “The @CNIL believes that the use of @googleanalytics is a violation of #GDPR”

  Immediate action required for Google Analytics users

  The CNIL and other EU-based data protection authorities have made their stance on Google Analytics clear and inaction will likely result in fines, which under the GDPR, can be up to €20 million or 4% of the organisation’s global turnover – whichever is higher.

  Based on the CNIL’s formal notice to the model case’s website manager, Google Analytics users should take immediate action to remove any chances of personal data being transferred to the US or find a Google Analytics alternative that is GDPR compliant. 

  

  English translation : “The CNIL considers that the use of Google Analytics is a violation of the GDPR. I use @matomo_org and I welcome it *winking face* It will squeal tires among growthackers who are slaughtering. Opportunity to look at alternative tools”

  Ready to begin your journey to GDPR compliance with Matomo ? Start your 21-day free trial now (no credit card required) and take advantage of our Google Analytics importer so you don’t lose any of your historical data. 

  Try 21-day free Cloud trial

  What does this mean for Matomo users ?

  As the GDPR continues to evolve, our users can rest assured that Matomo will be at the forefront of these changes. With Matomo Cloud, all data is stored in the EU or in your country of choice when you self-host on your own servers with Matomo On-Premise.

  Conclusion

  Google is in the EU’s crosshairs and organisations that continue to use their tools will be the one’s left to clean up the mess – not Google. Now is the time to act. Search for a Google Analytics alternative and close your compliance gaps today. 

  Join over 1 million other websites using Matomo now. Give Matomo a try with a 21-day free trial – no credit card required. 

  Try 21-day free Cloud trial

  We’d like to also bring attention to the privacy-fighting efforts from noyb and Max Schrems, as this should not go unnoticed. noyb is an independent, non-profit organisation that relies on the support of individuals. Support privacy by supporting noyb – donate or become a member now. 

  Contact details for media :

  For quotes or interviews, please email marketing@matomo.org

• Can't playback huge video which was uploaded to google storage. I Get Error : Unable to retrieve manifest /stream.m3u8 -Error : Could not retrieve file

  18 juillet 2022, par Dmytro Petskovych

  I upload file to google storage using "@ffmpeg-installer/ffmpeg" and @google-cloud/storage in my node.js App.
Step 1. file uploading to fs is in child processes - one process for each type of resolution (totaly six).
step 2. encription (converting to stream)
step 3. upload to google storage

  


  This way is working fine only with small file. When i upload large file I Get server Error : Unable to retrieve manifest /stream.m3u8 or Unable to retrieve .

  


  It seems that not all splits are uploaded to the cloud. but i checked they are there.

  


  I am currently using "Upload a directory to a bucket" in order to send the video from the client to the Google Cloud Storage bucket.

  


  when I upload video, actually I upload six videos, one for each type resolution

  


  for example when I upload video with duration one hour it split on chunk and totally I get more three thousands files.

  


  not all of this files are accessible when i try playback video. that's why I can't play video in all types of resolution and get error like

  


  maybe someone had the similar problem and helps fix it
.
i have no idea why i get that behavior only with huge file