Web Scraping part2: Digging deeper

Slides from the second web scraping through R session: Web scraping for the humanities and social sciences

Slides from the first session here

...the third session here

... and the fourth and final session here


In which we make sure we are comfortable with functions, before looking at XPath queries to download data from newspaper articles. Examples including BBC news and Guardian comments

Download the .Rpres file to use in Rstudio here

A regular R script with the code only can be accessed here

UPDATE March 2015:
New 2015 version of slides here
PDFs of slides available here

16 comments:

  1. Thank you for sharing, much appreciated.

    ReplyDelete
    Replies
    1. KARNA RASA HATI YANG GEMBIRA BERKAT BANTUAN AKI SOLEH
      MAKANYA SENGAJA NAMA BELIAU SAYA CANTUNKAN DI INTERNET !!!

      assalamualaikum wr, wb, saya IBU PUSPITA WATI saya Mengucapkan banyak2
      Terima kasih kepada: AKI SOLEH
      atas nomor togelnya yang kemarin AKI berikan "4D"
      alhamdulillah ternyata itu benar2 tembus AKI
      dan berkat bantuan AKI SOLEH saya bisa melunasi semua hutan2…
      orang tua saya yang ada di BANK BRI dan bukan hanya itu AKI alhamdulillah,
      sekarang saya sudah bisa bermodal sedikit untuk mencukupi kebutuhan keluarga saya sehari2.
      Itu semua berkat bantuan AKI SOLEH sekali lagi makasih banyak ya, AKI
      yang ingin merubah nasib
      seperti saya...?
      SILAHKAN GABUNG SAMA AKI SOLEH No; { 082-313-336-747 }

      Sebelum Gabung Sama AKI Baca Duluh Kata2 Yang Dibawah Ini
      Apakah anda termasuk dalam kategori di bawah ini...!!
      1: Di kejar2 tagihan hutang..
      2: Selaluh kalah dalam bermain togel
      3: Barang berharga sudah
      terjual buat judi togel..
      4: Sudah kemana2 tapi tidak
      menghasilkan, solusi yang tepat..!
      5: Sudah banyak dukun ditempati minta angka ritual blom dapat juga,
      satu jalan menyelesaikan masalah anda..
      Dijamin anda akan berhasil
      silahkan buktikan sendiri
      Atau Chat/Tlpn di WhatsApp (WA)
      No WA Aki : 082313336747

      TERIMA KASIH YANG PUNYA
      ROOM ATAS TUMPANGANYA SALAM KOMPAK SELALU
      "KLIK DISINI BOCORAN TOGEL SGP HK SDY DAN DLL"

      Delete
    2. KARNA RASA HATI YANG GEMBIRA BERKAT BANTUAN AKI SOLEH
      MAKANYA SENGAJA NAMA BELIAU SAYA CANTUNKAN DI INTERNET !!!

      assalamualaikum wr, wb, saya IBU PUSPITA WATI saya Mengucapkan banyak2
      Terima kasih kepada: AKI SOLEH
      atas nomor togelnya yang kemarin AKI berikan "4D"
      alhamdulillah ternyata itu benar2 tembus AKI
      dan berkat bantuan AKI SOLEH saya bisa melunasi semua hutan2…
      orang tua saya yang ada di BANK BRI dan bukan hanya itu AKI alhamdulillah,
      sekarang saya sudah bisa bermodal sedikit untuk mencukupi kebutuhan keluarga saya sehari2.
      Itu semua berkat bantuan AKI SOLEH sekali lagi makasih banyak ya, AKI
      yang ingin merubah nasib
      seperti saya...?
      SILAHKAN GABUNG SAMA AKI SOLEH No; { 082-313-336-747 }

      Sebelum Gabung Sama AKI Baca Duluh Kata2 Yang Dibawah Ini
      Apakah anda termasuk dalam kategori di bawah ini...!!
      1: Di kejar2 tagihan hutang..
      2: Selaluh kalah dalam bermain togel
      3: Barang berharga sudah
      terjual buat judi togel..
      4: Sudah kemana2 tapi tidak
      menghasilkan, solusi yang tepat..!
      5: Sudah banyak dukun ditempati minta angka ritual blom dapat juga,
      satu jalan menyelesaikan masalah anda..
      Dijamin anda akan berhasil
      silahkan buktikan sendiri
      Atau Chat/Tlpn di WhatsApp (WA)
      No WA Aki : 082313336747

      TERIMA KASIH YANG PUNYA
      ROOM ATAS TUMPANGANYA SALAM KOMPAK SELALU
      "KLIK DISINI BOCORAN TOGEL SGP HK SDY DAN DLL"

      Delete
  2. A solid and good set of articles you posted. Its too bad the urls respond with "Server overloaded, please throttle your requests."

    ReplyDelete
  3. The wiki one in the first set of slides overloads all the time. I was able to follow the class with some patience though. Good stuff, thanks for this.

    ReplyDelete
  4. Very clear n easy instructions.Thanks a ton...now i can scrape lyrics for eminem ;)

    ReplyDelete
  5. Thank you for very clear tutorial. I am having one problem with the
    "Example with dataframe" slide in the unlist function. it skips all NULL
    or blank values which gives error when building the data frame.
    I am still new to R and I can't figure out how to fix.
    Any ideas?

    ReplyDelete
    Replies
    1. I have found that adding
      if (length(title) == 0) title=NA
      will fix the issue however using
      if (is.null(title)) title <- NA
      as shown in Part 2 will not work in all cases

      Delete
  6. This comment has been removed by the author.

    ReplyDelete
  7. Gambling gambling online casino you.
    Gclub The online casino game offers you the opportunity to enjoy online casino games like online casino games. Comfortable with a variety of games that are ready to open players all the time. Online games are risky for both mobile phones and PCs. Do not worry, the problem is that online gambling is not available. Players can access the site directly or install the application. Enjoy your favorite game with all the fun and excitement every time you want. Not only this, our online gambling games are designed to meet the needs of current gamblers. Because it can bet gain. Convenient channel Fast withdrawals and work teams that are open to serve you like a pro. Betting online gambling is easy with our online risk website. With the service of players all the time and we also have live broadcast from the casino Poipet. It will give players full enjoyment and reliability. Service is not disappointing, we certainly guarantee click here. Royal1688

    ReplyDelete
  8. Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

    teX-ai

    Sentiment Analysis Tool

    ReplyDelete
  9. Your article is very good. Is an article that provides knowledge for living quite well
    slotxo

    ReplyDelete