Importing social media content without an account. Part one - Tweets

Twitter and Facebook are popular proprietary social network, and it is common to display Tweets or Facebook posts on a website. There are several Drupal modules which can enable us to do that, but they usually require you to have an account on these social networks. This post is the first of a two-part series exploring how to render content from social media networks in Drupal without the requirement of a user account, and describes how you can get Tweets from either a Twitter account or a Twitter #hashtag. The second part will look at how to render posts from a public Facebook page into your Drupal website. In both cases without the need to have an account with either of these social networks.

Requirements

  • Tweeper: a Twitter to RSS web scraper. Only required for Twitter (in our example, but also required for some other social networks)
  • Feeds, Feeds Tamper, and some other related modules and their dependencies

Installation of Tweeper

In the past, Twitter had a public RSS or Atom feed for all accounts or #hashtags. Unfortunately, in June 2013 Twitter.com retired their API v1.0, so it's not possible to access a user timeline via RSS anymore, additionally it is now also mandatory to authenticate via OAuth to access this public information in JSON format. However, in our case we do not want to authenticate, as we do not have (or do not want to link our website to) a Twitter account. Therefore, we use Tweeper to scrape the Twitter website for the information we require.

Download Tweeper from http://git.ao2.it/tweeper.git/ and install into its own directory under /sites/all/libraries, so that we have all required files under /sites/all/libraries/tweeper. We do not want to provide a public Twitter scraper, so we add a .htaccess file in the Tweeper directory only permitting access from localhost and our servers IP address.

Order deny,allow
Deny from all
Allow from 127.0.0.1
Allow from ::1
Allow from [IP4 address of server]
Allow from [IP6 address of server]

We now should have a working Tweeper installation at http://example.com/sites/all/libraries/tweeper, which is only accessible from our own server. We will use this to import Tweets.

Install Feeds and related modules

Next we install the modules Feeds, Feeds Tamper, Feeds Tamper PHP and their dependencies, and enable all of them (including Feeds UI and Feeds Tamper UI). This can be done via a simple drush command or manually via the "modules" admin page.

Prepare your content types

Twitter posts do not have a title, but Drupal nodes always need to have a title. So in this case we will use the title field for another purpose.

 

For Twitter, we also need two content types: one that is used for the Twitter feeds, and another one to store the Tweets itself. First, create a content type (which I called Tweets with the following fields:

  • Title: we use this for the Twitter username
  • Tweet: we rename the body field to Tweet, as that is where we store the tweet itself.

Now we are ready to configure feeds to import our Tweets. First we create the content type for our Twitter feeds (which I called Twitter Feeds), with the following fields:

  • Title: We can give our Twitter feed a meaningful title
  • Feed. This is a field of content type feeds module form element and is automatically created when you attach a new Feeds Importer to this content type (see below). You can't create this field manually.

You do not need a Body field, unless you want to have the option to add a more detailed description to your feed item.

Create your Twitter feed importer

Go to admin/structure/feeds to create your feed importer for your Twitter feeds. Click on Add importer and enter the name of your importer (I called it Twitter feed importer) and a description, and click on Create. Now we need to configure the importer for our content types.

Basic Settings

 

Click on Settings. Under Attach to content type, select the Twitter Feeds content type you created earlier. Next you need to adjust the settings under Node processor. Under Bundle select the content type your created for the individual tweets (Tweets). Check Update existing nodes, under Text format select Filtered HTML (or a custom text format accessible by the anonymous user), and uncheck Authorize. Under Expire nodes choose a timeframe that makes sense to you - if this is a busy Twitter account or #hashtag, a few days might be sufficient. For less busy accounts 1 month might be appropriate. Click Save

The next part are the Mappings. We map Title to Title, Published date to Published date, Item URL to URL (and marked as unique), Item GUID to GUID (also used as unique), and Description to Tweet. Save these mappings.

Now we need to manipulate the data we get via the feed - we need to tamper with it. Go to the Tamper tab at top. Under Title -> Title, our first mapping. Remember that Tweets do not have a title, so we use this field to store the Twitter user that posted the Tweet. The feed item has the format :, so we add a plugin: we select the plugin Execute PHP code, add a meaningful description (I used Get Twitter user), and enter the following PHP code:

 

$parts = explode(':', $field);
return $parts[0];

 

 

Under Description -> Tweet we need to add several plugins:

We got the Twitter user into the title field, so we do not want it here. So we add a plugin and again select Execute PHP code. We add a meaningful name, this time we also need to change the system name (as the default is php, which already exists), and add the following php code:

$substring=explode(':', $field, 2);
return $substring[1];

We are doing the same as under title, only now we save the second part of the exploded string.

We also need to add another plugin: Make URLs absolute. Just add it - there is nothing to configure here. This is needed to make sure that links (to #hashtags or Twitter users) link to Twitter, and not to non-existent local URLs.

Adding a Twitter feed

We can now add either a feed for a Twitter account or a Twitter #hashtag - in fact, we can add as many feeds as we want. Go to Add Content and select the Twitter Feed content type you created earlier. You need to add a title, and under URL you add the following URL

 

You can add as many feeds as you want - which is another advantage here. However, you still have no way to display them nicely. To achieve this, we need to use Views.

Adding a view to display your Tweets

You probably want a Block display of your Tweets. So create a view of content types Tweet with a block display, and display of fields. You need to add the following fields (all excluded from display) in the following order:

  • Feeds Item: Item URL: This holds the URL to the Twitter post on Twitter.
  • Title: Remember: this holds the Twitter username. We do rewrite this field to putput it as a link, and add the above field ([url]) as link path, and also tick the External server box
  • Body (or Tweets): This holds the tweet itself.
  • Post date: This holds the date of the tweet.

Last we add a Global: Custom text field which we use to display the Tweet. We add the following:

(<a href="[url]">@[title]</a>) on <a href="[url]">[created]</a>: [body]

Save the view and are now able to position the block where we want it in the site layout.

This works fine when you only have one Twitter feed. If you have more than one, this view would display all tweets, independent from which Twitter feed they originate. To filter by Twitter feed, take a note of the nid of each feed (Twitter Feed node), and add a relationship to Feed Items: Owner feed. This allows you to then add a filter on Feed item: Owner feed nid, so that you can only display the Twitter feeds in the block that you're after.

In the next part of this series we will look at importing posts from public Facebook pages. Stay tuned.