Web-Scraping: the Basics

Slides from the first session of my course about web scraping through R: Web scraping for the humanities and social sciences

Includes an introduction to the paste function, working with URLs, functions and loops.
Putting it all together we fetch data in JSON format about Wikipedia page views from http://stats.grok.se/

Solutions here:

Download the .Rpres file to use in Rstudio here

Slides from part two can be seen here

Slides from part three here

Slides from the fourth and final session here

UPDATE March 2015:
New 2015 version of slides here
PDFs of slides available here


Slides from part two can be seen here
Labels: , ,

22 comments:

  1. scagcurFebruary 20, 2014 at 8:20 AM

    Here is a shiny app that covers one of your examples
    http://glimmer.rstudio.com/pssguy/wikiSearchRates/

    ReplyDelete
    Replies
    1. ibupuspitawatiMarch 23, 2020 at 4:20 PM

      KARNA RASA HATI YANG GEMBIRA BERKAT BANTUAN AKI SOLEH
      MAKANYA SENGAJA NAMA BELIAU SAYA CANTUNKAN DI INTERNET !!!

      assalamualaikum wr, wb, saya IBU PUSPITA WATI saya Mengucapkan banyak2
      Terima kasih kepada: AKI SOLEH
      atas nomor togelnya yang kemarin AKI berikan "4D"
      alhamdulillah ternyata itu benar2 tembus AKI
      dan berkat bantuan AKI SOLEH saya bisa melunasi semua hutan2…
      orang tua saya yang ada di BANK BRI dan bukan hanya itu AKI alhamdulillah,
      sekarang saya sudah bisa bermodal sedikit untuk mencukupi kebutuhan keluarga saya sehari2.
      Itu semua berkat bantuan AKI SOLEH sekali lagi makasih banyak ya, AKI
      yang ingin merubah nasib
      seperti saya...?
      SILAHKAN GABUNG SAMA AKI SOLEH No; { 082-313-336-747 }

      Sebelum Gabung Sama AKI Baca Duluh Kata2 Yang Dibawah Ini
      Apakah anda termasuk dalam kategori di bawah ini...!!
      1: Di kejar2 tagihan hutang..
      2: Selaluh kalah dalam bermain togel
      3: Barang berharga sudah
      terjual buat judi togel..
      4: Sudah kemana2 tapi tidak
      menghasilkan, solusi yang tepat..!
      5: Sudah banyak dukun ditempati minta angka ritual blom dapat juga,
      satu jalan menyelesaikan masalah anda..
      Dijamin anda akan berhasil
      silahkan buktikan sendiri
      Atau Chat/Tlpn di WhatsApp (WA)
      No WA Aki : 082313336747

      TERIMA KASIH YANG PUNYA
      ROOM ATAS TUMPANGANYA SALAM KOMPAK SELALU
      "KLIK DISINI BOCORAN TOGEL SGP HK SDY DAN DLL"

      Delete
    2. ibupuspitawatiMarch 24, 2020 at 4:27 PM

      KARNA RASA HATI YANG GEMBIRA BERKAT BANTUAN AKI SOLEH
      MAKANYA SENGAJA NAMA BELIAU SAYA CANTUNKAN DI INTERNET !!!

      assalamualaikum wr, wb, saya IBU PUSPITA WATI saya Mengucapkan banyak2
      Terima kasih kepada: AKI SOLEH
      atas nomor togelnya yang kemarin AKI berikan "4D"
      alhamdulillah ternyata itu benar2 tembus AKI
      dan berkat bantuan AKI SOLEH saya bisa melunasi semua hutan2…
      orang tua saya yang ada di BANK BRI dan bukan hanya itu AKI alhamdulillah,
      sekarang saya sudah bisa bermodal sedikit untuk mencukupi kebutuhan keluarga saya sehari2.
      Itu semua berkat bantuan AKI SOLEH sekali lagi makasih banyak ya, AKI
      yang ingin merubah nasib
      seperti saya...?
      SILAHKAN GABUNG SAMA AKI SOLEH No; { 082-313-336-747 }

      Sebelum Gabung Sama AKI Baca Duluh Kata2 Yang Dibawah Ini
      Apakah anda termasuk dalam kategori di bawah ini...!!
      1: Di kejar2 tagihan hutang..
      2: Selaluh kalah dalam bermain togel
      3: Barang berharga sudah
      terjual buat judi togel..
      4: Sudah kemana2 tapi tidak
      menghasilkan, solusi yang tepat..!
      5: Sudah banyak dukun ditempati minta angka ritual blom dapat juga,
      satu jalan menyelesaikan masalah anda..
      Dijamin anda akan berhasil
      silahkan buktikan sendiri
      Atau Chat/Tlpn di WhatsApp (WA)
      No WA Aki : 082313336747

      TERIMA KASIH YANG PUNYA
      ROOM ATAS TUMPANGANYA SALAM KOMPAK SELALU
      "KLIK DISINI BOCORAN TOGEL SGP HK SDY DAN DLL"

      Delete
  2. AnonymousFebruary 21, 2014 at 12:44 AM

    Very nice, thanks. Hope to see the material from the other classes as well.

    ReplyDelete
  3. IwebMarch 31, 2014 at 12:24 AM

    Thanks for sharing the nice information with us and you have incredible work in this blog and i have to sure bookmark this blog .

    data scraping services

    ReplyDelete
  4. UnknownNovember 26, 2014 at 8:18 PM

    Thank you for sharing this awesome material!!!

    I am a very newbie in web scrapping, but I want to analize my spotify data, can you adress me to anywhere to achieve this goal?? I dont find any use of these APIs in R.

    ReplyDelete
  5. deevybeeMarch 14, 2015 at 12:29 AM

    Many thanks for this. Just in case others hit the same issues:
    When I pasted in your code from slides, it did not work because the quotes pasted as curly quotes. All fine when turned into straight quotes.

    I am using R2.15.3.
    rjson is not compatible with this, but RJSONIO works fine

    ReplyDelete
  6. AndrianavalonaMarch 21, 2015 at 3:59 AM

    Thanks for sharing.
    Very nice work.

    Tantely

    ReplyDelete
  7. UnknownMarch 25, 2015 at 12:59 PM

    Finally I understand some about this, thank you very much !!!!

    Regards from Mexico =")

    ReplyDelete
  8. UnknownMarch 9, 2016 at 11:55 PM

    10 Diet Myths That Circulating Around Us since Our Birth
    When it comes to weight loss, everyone seems to have some advice. But how do you separate fact from fiction?
    http://www.ehealthharmony.com/2016/03/08/diet-myths-circulating-around-us/

    ehealth

    ReplyDelete
  9. UnknownMay 31, 2016 at 12:44 AM

    Web Scraping Company provides web scraping, data scraping, website scraping, web data extraction, big data service, big data solution and data mining services. We provides any kind of data from any online web resource.

    ReplyDelete
  10. BrendaPalmeriJune 1, 2016 at 11:19 PM

    Wow amazing, Nice content I found so many interesting stuff in your blog especially its discussion Thanks to sharing thanks!
    web data extraction tools

    ReplyDelete
  11. AnonymousDecember 30, 2016 at 10:46 PM

    I am expecting more interesting topics from you. And this was nice content and definitely it will be useful for many people.school prospectus design uk

    ReplyDelete
  12. BrendaPalmeriJuly 29, 2017 at 3:56 AM

    I really appreciate your post and you explain each and every point very well. Thanks for sharing this information. And I’ll love to read your next post too. growth hacking

    ReplyDelete
  13. UnknownMay 29, 2018 at 9:28 PM

    Thanks for sharing the descriptive information on Python course. It’s really helpful to me since I'm taking Python training. Keep doing the good work and if you are interested to know more on Python, do check this Python tutorial.https://www.youtube.com/watch?v=qgOXopu4n7c

    ReplyDelete
  14. UnknownMay 29, 2018 at 9:28 PM

    Thanks for sharing the descriptive information on Python course. It’s really helpful to me since I'm taking Python training. Keep doing the good work and if you are interested to know more on Python, do check this Python tutorial.https://www.youtube.com/watch?v=qgOXopu4n7c

    ReplyDelete
  15. Steve BerkeOctober 6, 2019 at 6:33 PM

    I enjoyed reading your article. Please make more interesting topics like this on.
    I'll come back for more :)

    From Japs a researcher from AOC, a company whois into ecommerce website design Sydney

    ReplyDelete
  16. Hir InfotechNovember 18, 2019 at 10:10 PM

    Hey, thanks for providing the sharing information of web scraping. I’ll mention one more data scraping company: HIR INFOTECH PVT LTD.(https://hirinfotech.com/); we provide high-quality structured data to enhance business outcomes and enable intelligent decision making.

    We are a leader in the web scraping - data extraction industry. We provide high-quality, priority delivery on web data scraping, email scraping, product scraping, web searching, contact scraping, business directory extraction, and screen scraping requirements. We provide its clients with quality and accurate data and it’s affordable too! You can email us on hirinfotechcontactus@gmail.com or skype on live:d573055022082ed2

    ReplyDelete
  17. SagaminghahahaFebruary 10, 2020 at 10:23 PM

    Thank you for good and useful comments.
    สล็อตโจ๊กเกอร์

    ReplyDelete
  18. Lisa Math TeacherApril 4, 2021 at 11:01 AM

    Hi there,

    Very nice post and blog, I found it very explanatory and informative, thank you very much for sharing your knowledge and wisdom with us, we know how important is experience in our lives.

    take care and stay positive

    Your follower,

    Lisa from Concessionárias

    ReplyDelete
  19. FyndhereJune 23, 2021 at 11:31 PM

    Fyndhere Find Nearby Stores Bargain & Buy Business Listing

    Find nearby stores and local stores with Fyndhere Bargain with vendors and buy at discount List your business on Fyndhere

    https://www.fyndhere.com/

    ReplyDelete
  20. RHemaNovember 27, 2022 at 9:25 PM

    Amazing post. Thanks for the detailed slides. Your presentation is to the point. The information you gave on Web Scraping is impressive. It shows how knowledgeable you are on this topic. I appreciate your effort. Keep sharing more beautiful posts on the latest updates. I am foreseeing to learn.
    Courses after bcom

    ReplyDelete