Recherche avancée

Sur d’autres sites (7815)

• Cloaked Archive Wiki

  16 mai 2011, par Multimedia Mike — General

  Google’s Chrome browser has made me phenomenally lazy. I don’t even attempt to type proper, complete URLs into the address bar anymore. I just type something vaguely related to the address and let the search engine take over. I saw something weird when I used this method to visit Archive Team’s site :

  
  
  

  There’s greater detail when you elect to view more results from the site :

  
  
  

  As the administrator of a MediaWiki installation like the one that archiveteam.org runs on, I was a little worried that they might have a spam problem. However, clicking through to any of those out-of-place pages does not indicate anything related to pharmaceuticals. Viewing source also reveals nothing amiss.

  I quickly deduced that this is a textbook example of website cloaking. This is when a website reports different content to a search engine than it reports to normal web browsers (humans, presumably). General pseudocode :

  PLAIN TEXT

  C :

  1. if (web_request.user_agent_string == CRAWLER_USER_AGENT)
  2.  return cloaked_data ;
  3. else
  4.  return real_data ;

  You can verify this for yourself using the wget command line utility :

  <br />
$ wget --quiet --user-agent="<strong>Mozilla/5.0</strong>" \<br />
 http://www.archiveteam.org/index.php?title=Geocities -O - | grep \&lt;title\&gt;<br />
&lt;title&gt;GeoCities - Archiveteam&lt;/title&gt;

  $ wget —quiet —user-agent="Googlebot/2.1"
  http://www.archiveteam.org/index.php?title=Geocities -O - | grep \<title\>
  <title>Cheap xanax | Online Drug Store, Big Discounts</title>
  

  I guess the little web prank worked because the phaux-pharma stuff got indexed. It makes we wonder if there’s a MediaWiki plugin that does this automatically.

  For extra fun, here’s a site called the CloakingDetector which purports to be able to detect whether a page employs cloaking. This is just one humble observer’s opinion, but I don’t think the site works too well :
  
  

  
  
  

• Revision 31957 : légère amélioration ... Un peu de documentation dans le code également

  7 octobre 2009, par kent1@… — Log

  légère amélioration ...
  Un peu de documentation dans le code également

• How to take screenshots of streamers using the Twitch API

  10 décembre 2020, par oo92

  My goal is to create a dataset of gameplays on Twitch using the API. How I want to do it is this :

  &#xA;

  &#xA;
  1. Get a list of live streams using the API.

  &#xA;

  2. Use streamlink and ffmpeg on Python to take the screenshots through the Stream source

  &#xA;

  &#xA;

  To get the streams, I have the following code thanks to a SO user :

  &#xA;

  from twitch import TwitchClient&#xA;&#xA;client = TwitchClient(client_id=&#x27;<my client="client">&#x27;)&#xA;streams = client.streams.get_live_streams(limit=100)&#xA;&#xA;print(streams)&#xA;</my>

  &#xA;

  The output is this. Its a lot larger, I just made it shorter... :

  &#xA;

  [{'id': 40889579422, 'game': 'Among Us', 'broadcast_platform': 'live', 'community_id': '', 'community_ids': [], 'viewers': 74594, 'video_height': 1080, 'average_fps': 60, 'delay': 0, 'created_at': datetime.datetime(2020, 12, 9, 23, 53, 34), 'is_playlist': False, 'stream_type': 'live', 'preview': {'small': 'https://static-cdn.jtvnw.net/previews-ttv/live_user_sykkuno-80x45.jpg', 'medium': 'https://static-cdn.jtvnw.net/previews-ttv/live_user_sykkuno-320x180.jpg', 'large': 'https://static-cdn.jtvnw.net/previews-ttv/live_user_sykkuno-640x360.jpg', 'template': 'https://static-cdn.jtvnw.net/previews-ttv/live_user_sykkuno-{width}x{height}.jpg'}, 'channel': {'mature': False, 'status': 'amongus at 4 !!', 'broadcaster_language': 'en', 'broadcaster_software': '', 'display_name': 'Sykkuno', 'game': 'Among Us', 'language': 'en', 'id': 26154978, 'name': 'sykkuno', 'created_at': datetime.datetime(2011, 11, 15, 1, 29, 29, 140794), 'updated_at': datetime.datetime(2020, 12, 10, 0, 35, 50, 916363), 'partner': True, 'logo': 'https://static-cdn.jtvnw.net/jtv_user_pictures/sykkuno-profile_image-6ab1e70e07e29e9b-300x300.jpeg', 'video_banner': 'https://static-cdn.jtvnw.net/jtv_user_pictures/4b654ce5-58dc-4fa6-b77c-7250bb2d5269-channel_offline_image-1920x1080.png', 'profile_banner': 'https://static-cdn.jtvnw.net/jtv_user_pictures/1caee146-3323-4d45-9907-96c20c224d3e-profile_banner-480.png', 'profile_banner_background_color': '', 'url': 'https://www.twitch.tv/sykkuno', 'views': 23590965, 'followers': 1928724, 'broadcaster_type': '', 'description': 'Hi ! ', 'private_video': False, 'privacy_options_enabled': False}}, ... 


  &#xA;

  First, I want to know how I can iterate through the JSON to get the channel name using the id. I tried to do it on my own but it said that it isn't subscriptable.

  &#xA;

  Second, I have the following code that tries to use ffmpeg to get a screenshot of the stream through its source :

  &#xA;

  import streamlink, os

# This code connects to the streamer's source
# Get the Twitch API to work so you can add the streamer's name at the end of the link
username = 'Sykkuno'
streams = streamlink.streams('http://twitch.tv/' + username)

# Stream source
stream = streams["best"].url

# Directory where the screenshots will be saved
dir_path = os.getcwd() + '/' + username

# number of streamers
streamers = 1

os.system('ffmpeg -i ' + stream +' -r 0.5 -f image2 ${dir}/output_%09d.jpg')


  &#xA;

  But that is throwing the following error :

  &#xA;

  ffmpeg version 4.1.3-0ppa1~18.04 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version='0ppa1~18.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-nonfree --enable-libfdk-aac --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[hls,applehttp @ 0x557212854940] Opening 'https://video-edge-c2a360.yto01.abs.hls.ttvnw.net/v1/segment/CuAErHgNDEVK3cqjkRZLcz4El99YD2ZSy9tgtjc3ZTrPuN4584-GY3DKYO3MFCBt9M3v8_7IfHAlLvHeUn2wq4d-VC7u4Zv2-k1Bee9IPmIXbFQLZFKrYiT98OTzFMDaElsEZ9Sr4MXz6FoAC1yXhhZ0MYahZVnZa1Tuqnn217nN_rN8wr7kdScBIir_Uo1s2C_I8_54mi2uzdgxB9AYj0a0kG2UtdPkUVA6Qc7XIZ0nUhEeObEf80N0uW9ZU6WHpO3V6G9RD0VkmwUexDWk9MaLy1NJuvLSzRaMfOmKxhrzn3UDoY4CrQN11KOnHYCiCOfvhZmMKzSqtiA8YP0Q9iS03eZZhPQ3WxBHWhd6VeZ0btNeeoudGX73EBIj4ujZnKWKfPiN8K_HLj5FZWqQ5m_4Q1llQlnSAfmhzXR9PHAkz8nxRVcFR-tFokGzFEfkZGHngPNz9boLmo4KHx6404rocPUcXbTHkYsWXZwFC356AhfrNn6x0JYHGfcDpRsEFebLQljEpCyhnNOHFEf9b7Ipeiy521cCupzoEz1uMrzW77h9FJwn0GwvY3jp2KwRXJMvArYwiUHccBmfW7ZDLO9vH7eJGUnAzoy586KMSPLVeLxWMWRDfKkAcI8vYXOIuxNb_MZKm2O_dcoFrDDta5TZn-MLjFquT47P9NbXxlTTJfimlYyMG17SPoW3KJVtAJmjoj0GZ1ehftLJWz1Qgd8zmg40u4g8fz867eCHW_Tiv65yd6qJZD-_bD3G916fm3KeA9bPyhalTD99CWhQL7f0apCfX_6IiQElgPT6kOrFI-ISEG4GdhIpFi4ubzSx29pm7H4aDIXx6BAI5ziHVMPbrg.ts' for reading
[hls,applehttp @ 0x557212854940] Opening 'https://video-edge-c2a360.yto01.abs.hls.ttvnw.net/v1/segment/CuIECWLarz2I5hNCtGLSTDXbCIgahI91KOHXVmPHddzZediyiVbcicZpGakO99CJp1cK_6OnmOKBzZ3sn3KogNyPVRqSy32IpanlyPPUTz23TojfR5DTmJa3zampOzVMfETvvPEpj58-kGhQeQQGtK1zLV2h465RElCLkmc7SeWbXEZ5j6IQ38OVFZ0vdcMHVTNPtJaY7509bE196-9-5YYFiET_-kdkS2X6_lhVQBZrq45PBxApTzeLkDx9ZtGQx78dLr_tZepww_uVnJTwxI_TA3tU8z5_w8ml6rh0GK1W6lTlvuDkPKAwIDLvG736MtbPGz9cFPoRAFQxD3QSwAM-bO1grvWlNsOUDUYLXLNjuejmz8xmRpeE0pqJYxUboRZpxrPXTi52HcX8lpGT4Lx2z6hJcoi9npQttK58HDSHQ3cYH3rlNsYV_RlZ3F-u-fZSn8Em67-vAYeXAMaBe9xxv0Zu2n1TrdPICMyGmk-VgLK788IeDhxqM441GONGLxo6AF8RE5_OawTf9n_MVzImsX4LMn0oN8e8w6mjk7YDuNdA4mEl99Erg8xMdX6Q3fDYTdStaC-zQwXgMctfGpIsUpp91BtRBPMynFCxZ8fZB7NvGFNZTYWDvCZjisiwzs0N1pDdDM_Fkj3i46_Ou105z348PLHRA28Dt1qgn_NjzwRNoaowFx7PxQ-X2zRpAhwNcqaTOHYh5NYZHToVE16WiOe9HzVs_I_3Wqu44bypEsGrhxYhgXSGfvO757iTbErZHmkidsBF2BET5j3JeIifr4fJooalAKc_5uiSPMCGSTtV9hIQX5mQWJudg9UsMRp1AWVrnBoMQPAhy1UwBDtAOrBs.ts' for reading
Input #0, hls,applehttp, from 'https://video-weaver.yto01.hls.ttvnw.net/v1/playlist/CvADco9ZO_szpL7RVLVI3U6zl4IMo2uaUBvZTWAyEOKETN-SxI3m3hLLxpmCfz06rlyCGCPimEbk28kKPSbVYtMThiZFRtLxnrdpLr5DWBQwbpIStPQwMetU4Z04Mkm3nEWplL8hpn69Cmd_eWSdI1yHubK_sRL-n1ml5akAEWWGkS0OgVRQTTsYv4RpkbeWc8wrr3DsDBDfyrxciGSZevnStZOLwg-tNuNu3VNugLigNG1HMsNdxoJGO6wLyO-6FL5EQidOiSw2THSibAvADAWMVOxNX2z39d-nJwbVprbWFnox_lDlh-y88uhjj_MpU6OwkXP9vgrBDEmKjtRbYsIN5N94-8pDFEB3yIYk10vUiuod1yfqztVCqWo5m9r48uINP1CD-Bwgybo6K7Oko-TUjMj43GqAu0mmPtkgPFO69LpQQMifZTn_XbVQ5UUfoCwyl-ljObfRE51aCvZP_dOTsUyqi_JRE8h-C3306aW1ISXwCs3YpnjDxJv6yKyWTktBvsN5NWMVzJg-_-MAtTmoy-w2ppbbOozLzyrGF4zfCetaHLmHtbsKe1Ed1nndJg9b6T7v-87ExrI0eQwRjH3gmBTgqu_peS5oQGBaDOe4QSGMNFgf8lQfuqvFH-miXgpk8RytdorzT7wB05iX7c3bhBIQ3FvvjNIal3IgvyxKatZs4BoMW_P_CuyZOUJs2_Yr.m3u8':
  Duration: N/A, start: 60.000000, bitrate: N/A
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, unknown/bt709/unknown), 1920x1080, 29.97 tbr, 90k tbn, 2k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:2: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x557213e262c0] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/output_%09d.jpg':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: mjpeg, yuvj420p(pc), 1920x1080, q=2-31, 200 kb/s, 0.50 fps, 0.50 tbn, 0.50 tbc
    Metadata:
      variant_bitrate : 0
      encoder         : Lavc58.35.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
[image2 @ 0x557213012040] Could not open file : /output_000000001.jpg
av_interleaved_write_frame(): Input/output error
frame=    1 fps=0.0 q=7.5 Lsize=N/A time=00:00:02.00 bitrate=N/A speed=27.7x    
video:46kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!


  &#xA;