Logging into Twitter example

We will use the Twitter module as an example because there isn't anything unusual about their login process.

Prerequisites

Creating a module requires understanding HTML and some proficiency with a HTTP proxy. Before writing any code it is important to understand what we are trying to program. Perform/answer the following steps before developing the module:

• Create a username and password for the website
• Configure a browser to use a proxy tool, such as Burp Suite
• Capture a successful login request and a failed login request
• Determine a difference between the two. Typically this is either a response code or a string
• Send the successful login request with less headers and cookies, and determine the minimal parameters required. This is important because some sites require unusual headers for a login, such as a Referer header

With the above steps done writing the module will be straight forward.

Prerequisites for Twitter

Investigating Twitter in Burp Suite, I found the following answers to the prerequisite:

• The login page is www.twitter.com
• The login request requires a Content-Type header of application/x-www-form-urlencoded
• Login attempts always return a 302, a failed login contains the "login/error?username_or_email" string

Module development steps

The steps to create a module:

• Setting up blank module and test files
• Add it to the ModuleFactory
• Get initial cookies and login form
• Perform the login
• Check the login

Creating a blank module and test

Create an object that extends AbstractModule in the [modules package[(https://github.com/philwantsfish/shard/tree/master/src/main/scala/fish/philwants/modules)

The AbstractModule requires some fields and methods:

package fish.philwants.modules

import fish.philwants.Credentials

object TwitterModule extends AbstractModule {
  val uri = "https://twitter.com/"
  val moduleName = "Twitter"

  def tryLogin(creds: Credentials): Boolean = {
    true
  }
}

Create a test in the module test directory:

package fish.philwants

class TwitterModuleTest extends FlatSpec with Matchers {
  "Twitter module" should "detect a successful login" in {
    val creds = Credentials(TWITTER_USERNAME, TWITTER_PASSWORD)
    val mod = TwitterModule
    mod.tryLogin(creds) shouldBe true
  }

  it should "detect a failed login" in {
    val creds = Credentials(BAD_USERNAME_EMAIL, BAD_PASSWORD)
    val mod = TwitterModule
    mod.tryLogin(creds) shouldBe false
  }
}

The test DSL is provided by scalatest. The username and password should be added to the TestCredentials file.

Confirm the the module and test file are working with sbt "test-only *TwitterModule. You should see a test failure!

Add it to the ModuleFactory

Shard will only use modules that are created in the ModuleFactory. THis object just contains a list of module class names. Add the TwitterModule to the list.

Get initial cookies and login form

The login form contains the URI to login and required hidden form parameters. JSoup provides APIs to interact with forms. To get the form element first request the page with the form:

val resp = get(uri).execute()

The get API is provided by AbstractModule that provides defaults for timeouts and the User-Agent header.

The Shard framework provides 3 ways to parse forms from responses:

• Use the first form
• Find the form by id
• Find the form by class

The Twitter login form uses a class. We can select the form element by class and updating the username/password parameters:

val form = resp.selectForm("form.LoginForm.js-front-signin")
      .update("session[username_or_email]", creds.username)
      .update("session[password]", creds.password)

Perform the login

It is typical the login flow requires cookies. Try to login using the cookies from the initial request and the form.

val loginResp = form
      .submit()
      .cookies(resp.cookies())
      .header("Content-Type", "application/x-www-form-urlencoded")
      .followRedirects(false)
      .execute()

Check the login

During the prerequisites we found a string to identity failed logins. Check the response for this indicator:

val locationHeader = resp.header("Location")
!locationHeader.contains("login/error?username_or_email")

Testing

That is all! Once the two tests pass the module is ready for a pull request.