Recherche avancée
Médias (1)
-
The pirate bay depuis la Belgique
1er avril 2013, par
Mis à jour : Avril 2013
Langue : français
Type : Image
Autres articles (96)
-
MediaSPIP 0.1 Beta version
25 avril 2011, parMediaSPIP 0.1 beta is the first version of MediaSPIP proclaimed as "usable".
The zip file provided here only contains the sources of MediaSPIP in its standalone version.
To get a working installation, you must manually install all-software dependencies on the server.
If you want to use this archive for an installation in "farm mode", you will also need to proceed to other manual (...) -
Multilang : améliorer l’interface pour les blocs multilingues
18 février 2011, parMultilang est un plugin supplémentaire qui n’est pas activé par défaut lors de l’initialisation de MediaSPIP.
Après son activation, une préconfiguration est mise en place automatiquement par MediaSPIP init permettant à la nouvelle fonctionnalité d’être automatiquement opérationnelle. Il n’est donc pas obligatoire de passer par une étape de configuration pour cela. -
ANNEXE : Les plugins utilisés spécifiquement pour la ferme
5 mars 2010, parLe site central/maître de la ferme a besoin d’utiliser plusieurs plugins supplémentaires vis à vis des canaux pour son bon fonctionnement. le plugin Gestion de la mutualisation ; le plugin inscription3 pour gérer les inscriptions et les demandes de création d’instance de mutualisation dès l’inscription des utilisateurs ; le plugin verifier qui fournit une API de vérification des champs (utilisé par inscription3) ; le plugin champs extras v2 nécessité par inscription3 (...)
Sur d’autres sites (6547)
-
How to Check Website Traffic As Accurately As Possible
18 août 2023, par Erin — Analytics TipsIf you want to learn about the health of your website and the success of your digital marketing initiatives, there are few better ways than checking your website traffic.
It’s a great way to get a quick dopamine hit when things are up, but you can also use traffic levels to identify issues, learn more about your users or benchmark your performance. That means you need a reliable and easy way to check your website traffic over time — as well as a way to check out your competitors’ traffic levels, too.
In this article, we’ll show you how to do just that. You’ll learn how to check website traffic for both your and your competitor’s sites and discover why some methods of checking website traffic are better than others.
Why check website traffic ?
Dopamine hits aside, it’s important to constantly monitor your website’s traffic for several reasons.
Benchmark site performance
Keeping regular tabs on your traffic levels is a great way to track your website’s performance over time. It can help you plan for the future or identify problems.
For instance, growing traffic levels may mean expanding your business’s offering or investing in more inventory. On the flip side, decreasing traffic levels may suggest it’s time to revamp your marketing strategies or look into issues impacting your SEO.
Analyse user behaviour
Checking website traffic and user behaviour lets marketing managers understand how users interact with your website. Which pages are they visiting ? Which CTAs do they click on ? What can you do to encourage users to take the actions you want ? You can also identify issues that lead to high bounce rates and other problems.
The better you understand user behaviour, the easier it will be to give them what they want. For example, you may find that users spend more time on your landing pages than they do your blog pages. You could use that information to revise how you create blog posts or focus on creating more landing pages.
Improve the user experience
Once you understand how users behave on your website, you can use that information to fix errors, update your content and improve the user experience for the site.
You can even personalise the experience for customers, leading to significant growth. Research shows companies that grow faster derive 40% more of their revenue from personalisation.
That could come in the form of sweeping personalisations — like rearranging your website’s navigation bar based on user behaviour — or individual personalisation that uses analytics to transform sections or entire pages of your site based on user behaviour.
Optimise marketing strategies
You can use website traffic reports to understand where users are coming from and optimise your marketing plan accordingly. You may want to double down on organic traffic, for instance, or invest more in PPC advertising. Knowing current traffic estimates and how these traffic levels have trended over time can help you benchmark your campaigns and prioritise your efforts.
Increasing traffic levels from other countries can also help you identify new marketing opportunities. If you start seeing significant traffic levels from a neighbouring country or a large market, it could be time to take your business international and launch a cross-border campaign.
Filter unwanted traffic
A not-insignificant portion of your site’s traffic may be coming from bots and other unwanted sources. These can compromise the quality of your analytics and make it harder to draw insights. You may not be able to get rid of this traffic, but you can use analytics tools to remove it from your stats.
How to check website traffic on Matomo
If you want to check your website’s traffic, you’d be forgiven for heading to Google Analytics first. It’s the most popular analytics tool on the market, after all. But if you want a more reliable assessment of your website’s traffic, then we recommend using Matomo alongside Google Analytics.
The Matomo web analytics platform is an open-source solution that helps you collect accurate data about your website’s traffic and make more informed decisions as a result — all while enhancing the customer experience and ensuring GDPR compliance and user privacy.
Matomo also offers multiple ways to check website traffic :
Let’s look at all of them one by one.
The visits log report is a unique rundown of all of the individual visitors to your site. This offers a much more granular view than other tools that just show the total number of visitors for a given period.
You can access the visits log report by clicking on the reporting menu, then clicking Visitor and Visits Log. From there, you’ll be able to scroll through every user session and see the following information :
- The location of the user
- The total number of actions they took
- The length of time on site
- How they arrived at your site
- And the device they used to access your site
This may be overwhelming if your site receives thousands of visitors at a time. But it’s a great way to understand users at an individual level and appreciate the lifetime activity of specific users.
The Real-time visitor map is a visual display of users’ location for a given timeframe. If you have an international website, it’s a fantastic way to see exactly where in the world your traffic comes from.
You can access the Real-time Visitor Map by clicking Visitor in the main navigation menu and then Real-time Map. The map itself is colour-coded. Larger orange bubbles represent recent visits, and smaller dark orange and grey bubbles represent older visits. The map will refresh every five seconds, and new users appear with a flashing effect.
If you run TV or radio adverts, Matomo’s Real-time Map provides an immediate read on the effectiveness of your campaign. If your map lights up in the minutes following your ad, you know it’s been effective. It can also help you identify the source of bot attacks, too.
Finally, the Visits in Real-time report provides a snapshot of who is browsing your website. You can access this report under Visitors > Real-time and add it to your custom dashboards as a widget.
Open the report, and you’ll see the real-time flow of your site’s users and counters for visits and pageviews over the last 30 minutes and 24 hours. The report refreshes every five seconds with new users added to the top of the report with a fade-in effect.
The report provides a snapshot of each visitor, including :
- Whether they are new or a returning
- Their country
- Their browser
- Their operating system
- The number of actions they took
- The time they spent on the site
- The channel they came in from
- Whether the visitor converted a goal
3 other ways to check website traffic
You don’t need to use Matomo to check your website traffic. Here are three other tools you can use instead.
How to check website traffic on Google Analytics
Google Analytics is usually the first starting point for anyone looking to check their website traffic. It’s free to use, incredibly popular and offers a wide range of traffic reports.
Google Analytics lets you break down historical traffic data almost any way you wish. You can split traffic by acquisition channel (organic, social media, direct, etc.) by country, device or demographic.
It also provides real-time traffic reports that give you a snapshot of users on your site right now and over the last 30 minutes.
Google Analytics may be one of the most popular ways to check website traffic, but it could be better. Google Analytics 4 is difficult to use compared to its predecessor, and it also limits the amount of data you can track in accordance with privacy laws. If users refuse your cookie consent, Google Analytics won’t record these visits. In other words, you aren’t getting a complete view of your traffic by using Google Analytics alone.
That’s why it’s important to use Google Analytics alongside other web analytics tools (like Matomo) that don’t suffer from the same privacy issues. That way, you can make sure you track every single user who visits your site.
How to check website traffic on Google Search Console
Google Search Console is a free tool from Google that lets you analyse the search traffic that your site gets from Google.
The top-line report shows you how many times your website has appeared in Google Search, how many clicks it has received, the average clickthrough rate and the average position of your website in the search results.
Google Search Console is a great way to understand what you rank for and how much traffic your organic rankings generate. It will also show you which pages are indexed in Google and whether there are any crawling errors.
Unfortunately, Google Search Console is limited if you want to get a complete view of your traffic. While you can analyse search traffic in a huge amount of detail, it will not tell you how users who access your website directly or via social media behave.
How to check website traffic on Similarweb
Similarweb is a website analysis tool that estimates the total traffic of any site on the internet. It is one of the best tools for estimating how much traffic your competitors receive.
What’s great about Similarweb is that it estimates total traffic, not just traffic from search engines like many SEO tools. It even breaks down traffic by different channels, allowing you to see how your website compares against your competitors.
As you can see from the image above, Similarweb provides an estimate of total visits, bounce rate, the average number of pages users view per visit and the average duration on the site. The company also has a free browser extension that lets you check website traffic estimates as you browse the web.
You can use Similarweb for free to a point. But to really get the most out of this tool, you’ll need to upgrade to a premium plan which starts at $125 per user per month.
The price isn’t the only downside of using Similarweb to check the traffic of your own and your competitor’s websites. Ultimately, Similarweb is only an estimate — even if it’s a reasonably accurate one — and it’s no match for a comprehensive analytics tool.
7 website traffic metrics to track
Now that you know how to check your website’s traffic, you can start to analyse it. You can use plenty of metrics to assess the quality of your website traffic, but here are some of the most important metrics to track.
- New visitors : These are users who have never visited your website before. They are a great sign that your marketing efforts are working and your site is reaching more people. But it’s also important to track how they behave on the website to ensure your site caters effectively to new visitors.
- Returning visitors : Returning visitors are coming back to your site for a reason : either they like the content you’re creating or they want to make a purchase. Both instances are great. The more returning visitors, the better.
- Bounce rate : This is a measure of how many users leave your website without taking action. Different analytics tools measure this metric differently.
- Session duration : This is the length of time users spend on your website, and it can be a great gauge of whether they find your site engaging. Especially when combined with the metric below.
- Pages per session : This measures how many different pages users visit on average. The more pages they visit and the longer users spend on your website, the more engaging it is.
- Traffic source : Traffic can come from a variety of sources (organic, direct, social media, referral, etc.) Tracking which sources generate the most traffic can help you analyse and prioritise your marketing efforts.
- User demographics : This broad metric tells you more about who the users are that visit your website, what device they use, what country they come from, etc. While the bulk of your website traffic will come from the countries you target, an influx of new users from other countries can open the door to new opportunities.
Why do my traffic reports differ ?
If you use more than one of the methods above to check your website traffic, you’ll quickly realise that every traffic report differs. In some cases, the reasons are obvious. Any tool that estimates your traffic without adding code to your website is just that : an estimate. Tools like Similarweb will never offer the accuracy of analytics platforms like Matomo and Google Analytics.
But what about the differences between these analytics platforms themselves ? While each platform has a different way of recording user behaviour, significant differences in website traffic reports between analytics platforms are usually a result of how each platform handles user privacy.
A platform like Google Analytics requires users to accept a cookie consent banner to track them. If they accept, great. Google collects all of the data that any other analytics platform does. It may even collect more. If users reject cookie consent banners, however, then Google Analytics can’t track these visitors at all. They simply won’t show up in your traffic reports.
That doesn’t happen with all analytics platforms, however. A privacy-focused alternative like Matomo doesn’t require cookie consent banners (apart from in the United Kingdom and Germany) and can therefore continue to track visitors even after they have rejected a cookie consent screen from Google Analytics. This means that virtually all of your website traffic will be tracked regardless of whether users accept a cookie consent banner or not. And it’s why traffic reports in Matomo are often much higher than they are in Google Analytics.
Given that around half (47.32%) of adults in the European Union refuse to allow the use of personal data tracking for advertising purposes and that 95% of people will reject additional cookies when it is easy to do so, this means you could have vastly different traffic reports — and be missing out on a significant amount of user data.
If you’re serious about using web analytics to improve your website and optimise your marketing campaigns, then it is essential to use another analytics platform alongside Google Analytics.
Get more accurate traffic reports with Matomo
There are several methods to check website traffic. Some, like Similarweb, can provide estimates on your competitors’ traffic levels. Others, like Google Analytics, are free. But data doesn’t lie. Only privacy-focused analytics solutions like Matomo can provide accurate reports that account for every visitor.
Join over one million organisations using Matomo to accurately check their website traffic. Try it for free alongside GA today. No credit card required.
-
How to reduce size of moov atom of H.264 movies to improve streaming start for smartphones ?
12 mars 2023, par Andrey LebedevWe run a video service streaming movies to smartphones (iOS&Android).
We are encoding in H.264+AAC and using the mp4 container.
We have a problem that long movies (60 minutes+) take a very long time to
start playing and have tracked this down to the large size of moov
atom for these movies.
For 110 minute movies the atom is as large as 4.2Mb which obviously takes a long
time to download to a smart-phone over 3G !



Is there anyway to make the moov atom smaller ? We can reduce it bit
by dropping the audio sampling rate, but obviously anything below 22kHz
would not really be acceptable.



We are using ffmpeg as the encoder, and MP4Box to move the metadata
to the front of the file. Is there any way to get it to make
a smaller moov ? Any other encoders out there which make a smaller moov ?



For example...



Big size (280 Mb, 1h 49min) streamable mp4 (h.264, AAC) file have a big header size (4.2 Mb). File was encoded by two pass ffmpeg and MP4Box for replacing metadata into beginning of the file :



/usr/bin/ffmpeg -i /var/lib/encoder/incoming/2388 -aspect 320:210 -threads 8 -vcodec libx264 -profile baseline -level 13 -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -b:v 270k -maxrate 270k -bufsize 270k -g 30 -passlogfile /tmp/mediaservice/3100/video-IPH.ffmpeg -an -f rawvideo -pass 1 -y /dev/null

/usr/bin/ffmpeg -i /var/lib/encoder/incoming/2388 -aspect 320:210 -threads 8 -vcodec libx264 -profile baseline -level 13 -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -b:v 270k -maxrate 270k -bufsize 270k -g 30 -passlogfile /tmp/mediaservice/3100/video-IPH.ffmpeg -acodec libfaac -ac 2 -b:a 32k -ar 44100 -f mp4 -pass 2 -y /var/lib/encoder/encoded/3100/video-IPH.mp4

/usr/bin/MP4Box -quiet -tmp /tmp/mediaservice/3100/ -inter 500 /var/lib/encoder/encoded/3100/video-IPH.mp4



Media info (audio sample rate = 44100) :



General
Count : 278
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Count of video streams : 1
Count of audio streams : 1
Video_Format_List : AVC
Video_Format_WithHint_List : AVC
Codecs Video : AVC
Audio_Format_List : AAC
Audio_Format_WithHint_List : AAC
Audio codecs : AAC LC
Complete name : 1348645218_970458_2465.iph.mp4
File name : 1348645218_970458_2465.iph.mp4
File extension : mp4
Format : MPEG-4
Format : MPEG-4
Format/Extensions usually used : mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
Commercial name : MPEG-4
Format profile : Base Media
Internet media type : video/mp4
Codec ID : isom
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
Codec : MPEG-4
Codec : MPEG-4
Codec/Extensions usually used : mp4 m4v m4a m4b m4p 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma f4v
File size : 272703970
File size : 260 MiB
File size : 260 MiB
File size : 260 MiB
File size : 260 MiB
File size : 260.1 MiB
Duration : 6556027
Duration : 1h 49mn
Duration : 1h 49mn 16s 27ms
Duration : 1h 49mn
Duration : 01:49:16.027
Overall bit rate : 332767
Overall bit rate : 333 Kbps
Stream size : 4230761
Stream size : 4.03 MiB (2%)
Stream size : 4 MiB
Stream size : 4.0 MiB
Stream size : 4.03 MiB
Stream size : 4.035 MiB
Stream size : 4.03 MiB (2%)
Proportion of this stream : 0.01551
HeaderSize : 4230683
DataSize : 268473217
FooterSize : 70
IsStreamable : Yes
File last modification date : UTC 2012-09-26 12:38:19
File last modification date (local) : 2012-09-26 21:38:19
Writing application : Lavf54.6.100

Video
Count : 201
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
ID : 1
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : Baseline@L1.3
Format settings : 5 Ref Frames
Format settings, CABAC : No
Format settings, CABAC : No
Format settings, ReFrames : 5
Format settings, ReFrames : 5 frames
Format settings, GOP : M=1, N=30
Internet media type : video/H264
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
Codec : AVC
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec/Url : http://developers.videolan.org/x264.html
Codec/CC : avc1
Codec profile : Baseline@L1.3
Codec settings : 5 Ref Frames
Codec settings, CABAC : No
Codec_Settings_RefFrames : 5
Duration : 6556017
Duration : 01:49:16.017
Bit rate : 270000
Bit rate : 270 Kbps
Width : 480
Width : 480 pixels
Height : 270
Height : 270 pixels
Pixel aspect ratio : 1.000
Display aspect ratio : 1.778
Display aspect ratio : 16:9
Rotation : 0.000
Frame rate mode : CFR
Frame rate mode : Constant
FrameRate_Mode_Original : VFR
Frame rate : 29.970
Frame rate : 29.970 fps
Frame count : 196484
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Bits/(Pixel*Frame) : 0.070
Stream size : 220159060
Stream size : 210 MiB (81%)
Stream size : 210 MiB
Stream size : 210 MiB
Stream size : 210 MiB
Stream size : 210.0 MiB
Stream size : 210 MiB (81%)
Proportion of this stream : 0.80732
Writing library : x264 - core 125
Writing library : x264 core 125
Writing library/Name : x264
Writing library/Version : core 125
Encoding settings : cabac=0 / ref=5 / deblock=1:0:0 / analyse=0x1:0x131 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=8 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=30 / keyint_min=16 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=2pass / mbtree=1 / bitrate=270 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=270 / vbv_bufsize=270 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Tagged date : UTC 2012-09-25 07:21:37

Audio
Count : 169
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
ID : 2
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Commercial name : AAC
Format profile : LC
Codec ID : 40
Codec : AAC LC
Codec : AAC LC
Codec/Family : AAC
Codec/CC : 40
Duration : 6556027
Duration : 1h 49mn
Duration : 1h 49mn 16s 27ms
Duration : 1h 49mn
Duration : 01:49:16.027
Bit rate mode : VBR
Bit rate mode : Variable
Bit rate : 58955
Bit rate : 59.0 Kbps
Maximum bit rate : 270000
Maximum bit rate : 270 Kbps
Channel(s) : 2
Channel(s) : 2 channels
Channel positions : Front: L R
Channel positions : 2/0/0
Sampling rate : 44100
Sampling rate : 44.1 KHz
Samples count : 289120791
Compression mode : Lossy
Compression mode : Lossy
Stream size : 48314149
Stream size : 46.1 MiB (18%)
Stream size : 46 MiB
Stream size : 46 MiB
Stream size : 46.1 MiB
Stream size : 46.08 MiB
Stream size : 46.1 MiB (18%)
Proportion of this stream : 0.17717
Tagged date : UTC 2012-09-25 07:21:37



Moov atom info (/moov/trak[0] - video, /moov/trak[1] - audio) sample rate 44100 :
(look stsz and stts nodes in trak)



Atom ftyp @ 0 of size: 32, ends @ 32
Atom moov @ 32 of size: 4230651, ends @ 4230683
 Atom mvhd @ 40 of size: 108, ends @ 148
 Atom trak @ 148 of size: 868970, ends @ 869118
 Atom tkhd @ 156 of size: 92, ends @ 248
 Atom edts @ 248 of size: 36, ends @ 284
 Atom elst @ 256 of size: 28, ends @ 284
 Atom mdia @ 284 of size: 868834, ends @ 869118
 Atom mdhd @ 292 of size: 32, ends @ 324
 Atom hdlr @ 324 of size: 45, ends @ 369
 Atom minf @ 369 of size: 868749, ends @ 869118
 Atom vmhd @ 377 of size: 20, ends @ 397
 Atom dinf @ 397 of size: 36, ends @ 433
 Atom dref @ 405 of size: 28, ends @ 433
 Atom stbl @ 433 of size: 868685, ends @ 869118
 Atom stsd @ 441 of size: 149, ends @ 590
 Atom avc1 @ 457 of size: 133, ends @ 590
 Atom avcC @ 543 of size: 47, ends @ 590
 Atom stts @ 590 of size: 24, ends @ 614
 Atom stss @ 614 of size: 26340, ends @ 26954
 Atom stsc @ 26954 of size: 52, ends @ 27006
 Atom stsz @ 27006 of size: 785956, ends @ 812962
 Atom stco @ 812962 of size: 56156, ends @ 869118
 Atom trak @ 869118 of size: 3361468, ends @ 4230586
 Atom tkhd @ 869126 of size: 92, ends @ 869218
 Atom edts @ 869218 of size: 36, ends @ 869254
 Atom elst @ 869226 of size: 28, ends @ 869254
 Atom mdia @ 869254 of size: 3361332, ends @ 4230586
 Atom mdhd @ 869262 of size: 32, ends @ 869294
 Atom hdlr @ 869294 of size: 45, ends @ 869339
 Atom minf @ 869339 of size: 3361247, ends @ 4230586
 Atom smhd @ 869347 of size: 16, ends @ 869363
 Atom dinf @ 869363 of size: 36, ends @ 869399
 Atom dref @ 869371 of size: 28, ends @ 869399
 Atom stbl @ 869399 of size: 3361187, ends @ 4230586
 Atom stsd @ 869407 of size: 91, ends @ 869498
 Atom mp4a @ 869423 of size: 75, ends @ 869498
 Atom esds @ 869459 of size: 39, ends @ 869498
 **Atom stts @ 869498 of size: 2135816, ends @ 3005314**
 Atom stsc @ 3005314 of size: 39712, ends @ 3045026
 **Atom stsz @ 3045026 of size: 1129400, ends @ 4174426**
 Atom stco @ 4174426 of size: 56160, ends @ 4230586
 Atom udta @ 4230586 of size: 97, ends @ 4230683
 Atom meta @ 4230594 of size: 89, ends @ 4230683
 Atom hdlr @ 4230606 of size: 33, ends @ 4230639
 Atom ilst @ 4230639 of size: 44, ends @ 4230683
 Atom ©too @ 4230647 of size: 36, ends @ 4230683
 Atom data @ 4230655 of size: 28, ends @ 4230683
Atom mdat @ 4230683 of size: 268473217, ends @ 272703900
Atom free @ 272703900 of size: 8, ends @ 272703908
Atom free @ 272703908 of size: 62, ends @ 272703970
------------------------------------------------------
Total size: 272703970 bytes; 50 atoms total. AtomicParsley version: 0.9.0 (utf8)
Media data: 268473217 bytes; 4230753 bytes all other atoms (1.551% atom overhead).
Total free atom space: 70 bytes; 0.000% waste. Padding available: 0 bytes.
------------------------------------------------------



After reencoding this movie with audio sample rate 11025 header size much less :



Media info (audio sample rate = 11025) : (crop duplicate info)



General
***
HeaderSize : 1276359

Video
***

Audio
Count : 169
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
ID : 2
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Commercial name : AAC
Format profile : LC
Codec ID : 40
Codec : AAC LC
Codec : AAC LC
Codec/Family : AAC
Codec/CC : 40
Duration : 6556132
Duration : 1h 49mn
Duration : 1h 49mn 16s 132ms
Duration : 1h 49mn
Duration : 01:49:16.132
Bit rate mode : VBR
Bit rate mode : Variable
Bit rate : 37991
Bit rate : 38.0 Kbps
Maximum bit rate : 128000
Maximum bit rate : 128 Kbps
Channel(s) : 2
Channel(s) : 2 channels
Channel positions : Front: L R
Channel positions : 2/0/0
Sampling rate : 11025
Sampling rate : 11.025 KHz
Samples count : 72281355
Compression mode : Lossy
Compression mode : Lossy
Stream size : 31134257
Stream size : 29.7 MiB (12%)
Stream size : 30 MiB
Stream size : 30 MiB
Stream size : 29.7 MiB
Stream size : 29.69 MiB
Stream size : 29.7 MiB (12%)
Proportion of this stream : 0.12327
Tagged date : UTC 2012-09-25 13:20:28



Moov atom info (/moov/trak[0] - video, /moov/trak[1] - audio) sample rate 11025 :



Atom ftyp @ 0 of size: 32, ends @ 32
Atom moov @ 32 of size: 1276327, ends @ 1276359
 Atom mvhd @ 40 of size: 108, ends @ 148
 Atom trak @ 148 of size: 821662, ends @ 821810
 Atom tkhd @ 156 of size: 92, ends @ 248
 Atom edts @ 248 of size: 36, ends @ 284
 Atom elst @ 256 of size: 28, ends @ 284
 Atom mdia @ 284 of size: 821526, ends @ 821810
 Atom mdhd @ 292 of size: 32, ends @ 324
 Atom hdlr @ 324 of size: 45, ends @ 369
 Atom minf @ 369 of size: 821441, ends @ 821810
 Atom vmhd @ 377 of size: 20, ends @ 397
 Atom dinf @ 397 of size: 36, ends @ 433
 Atom dref @ 405 of size: 28, ends @ 433
 Atom stbl @ 433 of size: 821377, ends @ 821810
 Atom stsd @ 441 of size: 149, ends @ 590
 Atom avc1 @ 457 of size: 133, ends @ 590
 Atom avcC @ 543 of size: 47, ends @ 590
 Atom stts @ 590 of size: 24, ends @ 614
 Atom stss @ 614 of size: 26340, ends @ 26954
 Atom stsc @ 26954 of size: 52, ends @ 27006
 Atom stsz @ 27006 of size: 785956, ends @ 812962
 Atom stco @ 812962 of size: 8848, ends @ 821810
 Atom trak @ 821810 of size: 454452, ends @ 1276262
 Atom tkhd @ 821818 of size: 92, ends @ 821910
 Atom edts @ 821910 of size: 36, ends @ 821946
 Atom elst @ 821918 of size: 28, ends @ 821946
 Atom mdia @ 821946 of size: 454316, ends @ 1276262
 Atom mdhd @ 821954 of size: 32, ends @ 821986
 Atom hdlr @ 821986 of size: 45, ends @ 822031
 Atom minf @ 822031 of size: 454231, ends @ 1276262
 Atom smhd @ 822039 of size: 16, ends @ 822055
 Atom dinf @ 822055 of size: 36, ends @ 822091
 Atom dref @ 822063 of size: 28, ends @ 822091
 Atom stbl @ 822091 of size: 454171, ends @ 1276262
 Atom stsd @ 822099 of size: 91, ends @ 822190
 Atom mp4a @ 822115 of size: 75, ends @ 822190
 Atom esds @ 822151 of size: 39, ends @ 822190
 Atom stts @ 822190 of size: 161368, ends @ 983558
 Atom stsc @ 983558 of size: 1480, ends @ 985038
 Atom stsz @ 985038 of size: 282372, ends @ 1267410
 Atom stco @ 1267410 of size: 8852, ends @ 1276262
 Atom udta @ 1276262 of size: 97, ends @ 1276359
 Atom meta @ 1276270 of size: 89, ends @ 1276359
 Atom hdlr @ 1276282 of size: 33, ends @ 1276315
 Atom ilst @ 1276315 of size: 44, ends @ 1276359
 Atom ©too @ 1276323 of size: 36, ends @ 1276359
 Atom data @ 1276331 of size: 28, ends @ 1276359
Atom mdat @ 1276359 of size: 251293325, ends @ 252569684
Atom free @ 252569684 of size: 8, ends @ 252569692
Atom free @ 252569692 of size: 62, ends @ 252569754
------------------------------------------------------
Total size: 252569754 bytes; 50 atoms total. AtomicParsley version: 0.9.0 (utf8)
Media data: 251293325 bytes; 1276429 bytes all other atoms (0.505% atom overhead).
Total free atom space: 70 bytes; 0.000% waste. Padding available: 0 bytes.
------------------------------------------------------



On slow connection this movie start playing after 30-40 seconds until header info (4.2 Mb) downloading. I need that movie start playing fast as it possible. And i have next questions :



- 

-
How reduce size of movie header ?
-
How reduce size of
/moov[0]/trak[1]/mdia[0]/minf[0]/stbl[0] and why it so big when
sample rate 44100 ?






-
-
Parallelize Youtube video frame download using yt-dlp and cv2
4 mars 2023, par zulle99My task is to download multiple sequences of successive low resolution frames of Youtube videos.


I summarize the main parts of the process :


- 

- Each bag of shots have a dimension of half a second (depending on the current fps)
- In order to grab useful frames I've decided to remove the initial and final 10% of each video since it is common to have an intro and outro. Moreover
- I've made an array of pair of initial and final frame to distribute the load on multiple processes using
ProcessPoolExecutor(max_workers=multiprocessing.cpu_count()) - In case of failure/exception I completly remove the relative directory










The point is that it do not scale up, since while running I noticesd that all CPUs had always a load lower that the 20% more or less. In addition since with these shots I have to run multiple CNNs, to prevent overfitting it is suggested to have a big dataset and not a bounch of shots.


Here it is the code :


import yt_dlp
import os
from tqdm import tqdm
import cv2
import shutil
import time
import random
from concurrent.futures import ProcessPoolExecutor
import multiprocessing
import pandas as pd
import numpy as np
from pathlib import Path
import zipfile


# PARAMETERS
percentage_train_test = 50
percentage_bag_shots = 20
percentage_to_ignore = 10

zip_f_name = f'VideoClassificationDataset_{percentage_train_test}_{percentage_bag_shots}_{percentage_to_ignore}'
dataset_path = Path('/content/VideoClassificationDataset')

# DOWNOAD ZIP FILES
!wget --no-verbose https://github.com/gtoderici/sports-1m-dataset/archive/refs/heads/master.zip

# EXTRACT AND DELETE THEM
!unzip -qq -o '/content/master.zip' 
!rm '/content/master.zip'

DATA = {'train_partition.txt': {},
 'test_partition.txt': {}}

LABELS = []

train_dict = {}
test_dict = {}

path = '/content/sports-1m-dataset-master/original'

for f in os.listdir(path):
 with open(path + '/' + f) as f_txt:
 lines = f_txt.readlines()
 for line in lines:
 splitted_line = line.split(' ')
 label_indices = splitted_line[1].rstrip('\n').split(',') 
 DATA[f][splitted_line[0]] = list(map(int, label_indices))

with open('/content/sports-1m-dataset-master/labels.txt') as f_labels:
 LABELS = f_labels.read().splitlines()


TRAIN = DATA['train_partition.txt']
TEST = DATA['test_partition.txt']
print('Original Train Test length: ', len(TRAIN), len(TEST))

# sample a subset percentage_train_test
TRAIN = dict(random.sample(TRAIN.items(), (len(TRAIN)*percentage_train_test)//100))
TEST = dict(random.sample(TEST.items(), (len(TEST)*percentage_train_test)//100))

print(f'Sampled {percentage_train_test} Percentage Train Test length: ', len(TRAIN), len(TEST))


if not os.path.exists(dataset_path): os.makedirs(dataset_path)
if not os.path.exists(f'{dataset_path}/train'): os.makedirs(f'{dataset_path}/train')
if not os.path.exists(f'{dataset_path}/test'): os.makedirs(f'{dataset_path}/test')


Function to extract a sequence of continuous frames :


def extract_frames(directory, url, idx_bag, start_frame, end_frame):
 capture = cv2.VideoCapture(url)
 count = start_frame

 capture.set(cv2.CAP_PROP_POS_FRAMES, count)
 os.makedirs(f'{directory}/bag_of_shots{str(idx_bag)}')

 while count < end_frame:

 ret, frame = capture.read()

 if not ret: 
 shutil.rmtree(f'{directory}/bag_of_shots{str(idx_bag)}')
 return False

 filename = f'{directory}/bag_of_shots{str(idx_bag)}/shot{str(count - start_frame)}.png'

 cv2.imwrite(filename, frame)
 count += 1

 capture.release()
 return True


Function to spread the load along multiple processors :


def video_to_frames(video_url, labels_list, directory, dic, percentage_of_bags):
 url_id = video_url.split('=')[1]
 path_until_url_id = f'{dataset_path}/{directory}/{url_id}'
 try: 

 ydl_opts = {
 'ignoreerrors': True,
 'quiet': True,
 'nowarnings': True,
 'simulate': True,
 'ignorenoformatserror': True,
 'verbose':False,
 'cookies': '/content/all_cookies.txt',
 #https://stackoverflow.com/questions/63329412/how-can-i-solve-this-youtube-dl-429
 }
 ydl = yt_dlp.YoutubeDL(ydl_opts)
 info_dict = ydl.extract_info(video_url, download=False)

 if(info_dict is not None and info_dict['fps'] >= 20):
 # I must have a least 20 frames per seconds since I take half of second bag of shots for every video

 formats = info_dict.get('formats', None)

 # excluding the initial and final 10% of each video to avoid noise
 video_length = info_dict['duration'] * info_dict['fps']

 shots = info_dict['fps'] // 2

 to_ignore = (video_length * percentage_to_ignore) // 100
 new_len = video_length - (to_ignore * 2)
 tot_stored_bags = ((new_len // shots) * percentage_of_bags) // 100 # ((total_possbile_bags // shots) * percentage_of_bags) // 100
 if tot_stored_bags == 0: tot_stored_bags = 1 # minimum 1 bag of shots

 skip_rate_between_bags = (new_len - (tot_stored_bags * shots)) // (tot_stored_bags-1) if tot_stored_bags > 1 else 0

 chunks = [[to_ignore+(bag*(skip_rate_between_bags+shots)), to_ignore+(bag*(skip_rate_between_bags+shots))+shots] for bag in range(tot_stored_bags)]
 # sequence of [[start_frame, end_frame], [start_frame, end_frame], [start_frame, end_frame], ...]


 # ----------- For the moment I download only shots form video that has 144p resolution -----------

 res = {
 '160': '144p',
 '133': '240p',
 '134': '360p',
 '135': '360p',
 '136': '720p'
 }

 format_id = {}
 for f in formats: format_id[f['format_id']] = f
 #for res in resolution_id:
 if list(res.keys())[0] in list(format_id.keys()):
 video = format_id[list(res.keys())[0]]
 url = video.get('url', None)
 if(video.get('url', None) != video.get('manifest_url', None)):

 if not os.path.exists(path_until_url_id): os.makedirs(path_until_url_id)

 with ProcessPoolExecutor(max_workers=multiprocessing.cpu_count()) as executor:
 for idx_bag, f in enumerate(chunks): 
 res = executor.submit(
 extract_frames, directory = path_until_url_id, url = url, idx_bag = idx_bag, start_frame = f[0], end_frame = f[1])
 
 if res.result() is True: 
 l = np.zeros(len(LABELS), dtype=int) 
 for label in labels_list: l[label] = 1
 l = np.append(l, [shots]) # appending the number of shots taken in the list before adding it on the dictionary

 dic[f'{directory}/{url_id}/bag_of_shots{str(idx_bag)}'] = l.tolist()


 except Exception as e:
 shutil.rmtree(path_until_url_id)
 pass


Download of TRAIN bag of shots :


start_time = time.time()
pbar = tqdm(enumerate(TRAIN.items()), total = len(TRAIN.items()), leave=False)

for _, (url, labels_list) in pbar: video_to_frames(
 video_url = url, labels_list = labels_list, directory = 'train', dic = train_dict, percentage_of_bags = percentage_bag_shots)

print("--- %s seconds ---" % (time.time() - start_time))


Download of TEST bag of shots :


start_time = time.time()
pbar = tqdm(enumerate(TEST.items()), total = len(TEST.items()), leave=False)

for _, (url, labels_list) in pbar: video_to_frames(
 video_url = url, labels_list = labels_list, directory = 'test', dic = test_dict, percentage_of_bags = percentage_bag_shots)

print("--- %s seconds ---" % (time.time() - start_time))


Save the .csv files


train_df = pd.DataFrame.from_dict(train_dict, orient='index', dtype=int).reset_index(level=0)
train_df = train_df.rename(columns={train_df.columns[-1]: 'shots'})
train_df.to_csv('/content/VideoClassificationDataset/train.csv', index=True)

test_df = pd.DataFrame.from_dict(test_dict, orient='index', dtype=int).reset_index(level=0)
test_df = test_df.rename(columns={test_df.columns[-1]: 'shots'})
test_df.to_csv('/content/VideoClassificationDataset/test.csv', index=True)


Syndiquer tout le site
