Hello! We are MTR Design and site making is our speciality. We work with UK based startups, established businesses, media companies and creative individuals and turn their ideas into reality. This is a thrill and we totally love it.

SAP SuccessFactors SAML Authentication in Python

Author: Hristo Deshev

Working with SAML assertions and OAuth bearer tokens when using the SAP SuccessFactors API

We'll go over the SuccessFactors API authentication mechanism and how to make it play nice with Python. We will be using SAML assertions in our OAuth authentication and then use the OData API to read SuccessFactors entities. Fully-working source code and sample scripts included. read more ›

The SuccessFactors API gives us access to any data entity in the system with an easy to use interface. And I really mean easy -- the coolest thing about it is that it is based on the OData standard. OData is both simple to use in ad-hoc requests and there are a lot of client libraries out there that can make building queries easier. I usually prefer the former approach.

The OData specification does not specify an authentication and authorization mechanism, and the SuccessFactors team has decided to embrace another popular standard: OAuth 2.0 using the SAML bearer assertion flow. The SFSF SAML authentication story is not too different than the Jam one, but it has its own quirks. In a way it could be a lot easier to use SAML assertions with SuccessFactors, but that comes with a price - there is an associated security risk that I will help you avoid by doing some extra work.

The plan for this guide:

  • Configuring SuccessFactors for OAuth authentication.
  • Generating SAML assertions: ** Using the SFSF API. ** Using our own XML-signing code.
  • Obtaining an access token.
  • Authenticating OData requests.

OAuth Access Configuration

Unlike Jam, when working with SuccessFactors authentication, all you need to do is configure an OAuth client application. SFSF is smart enough to use it as an identity provider if you configure an X.509 certificate for your application.

Registering the application goes as following:

  • Generate a RSA key pair and export your public key as a X.509 certificate. Use the generate_keys.sh tool and consult the Jam SAML article if you get stuck.
  • Go to the SFSF AdminTools page and add a new OAuth client. Paste your X.509 certificate body in the textbox:

OAuth settings


Note: OpenSSL-generated certificates contain -----BEGIN CERTIFICATE-----/-----END CERTIFICATE----- text guards in their first and last lines respectively. The SuccessFactors admin seems to choke on those, so you need to remove the first and last lines. Just select the certificate body between those lines and paste it in the textbox above.

Generating the SAML assertion

There are two ways you can generate a SuccessFactors SAML assertion:

  • By using the SFSF assertion API.
  • By generating it and signing it yourself.

I'd like to take the moment and give you a warning against using the assertion API in production. First, there is the performance side of the story -- there is an extra server roundtrip involved every time you authenticate against the server which can get slow.

Even more important here are the security implications. In order to generate and sign a SAML assertion, the server needs access to your private key. Read that again -- you will be giving out your private key to someone else. That doesn't sit too well with me. I'd recommend that you try the API a couple of times to get a hold on the generated assertion document and then start generating that yourself.

Using the SuccessFactors Assertion API

With the above warning in place, let's get started talking to the assertion API. We need to issue a HTTP POST request to the /oauth/idp resource and pass the OAuth client id, our authenticated user ID, the access token generation URL (used as the next authentication step), and our specially formatted private key.

def get_assertion_from_sf(self):
    Send our private key to the SFSF IdP API and let it generate an assertion for us.
    Not ideal and incurs an additional API roundtrip. Use only for testing/debugging purposes.
    user_id = self.sf_user_id
    with open(self.private_key) as key_file:
        # remove ---BEGIN/---END lines (first and last)
        # strip whitespace and squash everything on a single line
        flattened_key = ''.join([l.strip() for l in key_file.readlines()[1:-1]])

    assertion_request = dict(
    response = requests.post(self.idp_url, data=assertion_request)
    return response.content

Note the key "flattening" logic above. We need to get rid of our first and last lines (containing the -----BEGIN RSA PRIVATE KEY-----/-----END RSA PRIVATE KEY----- markers), strip whitespace and squash everything in a single line.

The assertion we get back is a single-line base64-encoded XML document that we can just pass to the access code API (see below).

Generating an Assertion Ourselves

A quick base64-decode on the SAML document we get from the API above can show us how we can generate such a document ourselves. Here are the data items we need:

  • The recipient URL -- set to the SFSF OData URL.
  • The SAML audience string -- hardcoded to www.successfactors.com.
  • The authenticated user ID.
  • The OAuth application client ID.
  • A session id that doesn't matter too much. We hardcode it to mocksession.
  • Some timestamps: authentication instant and expiration times.

And now the XML template:

IssueInstant="{issue_instant}" Version="2.0"
    <saml2:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified">{user_id}</saml2:NameID>
    <saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">

      <saml2:SubjectConfirmationData NotOnOrAfter="{not_valid_after}"
      Recipient="{sf_root_url}/odata/v2" />
  <saml2:Conditions NotBefore="{not_valid_before}"
  <saml2:AuthnStatement AuthnInstant="{issue_instant}"
  <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
      <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315" />
      <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
      <Reference URI="">
          <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

We can now use an approach similar to the one used when generating and signing Jam assertions:

def get_local_assertion(self):
    Generate and sign the SAML assertion ourselves.
    user_id = self.sf_user_id

    unsigned_assertion = sf_saml.generate_assertion(
    signed = sf_saml.sign_assertion(unsigned_assertion, self.private_key)
    return signed.encode('base64').replace('\n', '')

The sf_saml module takes care of generating and signing assertions and is very similar to the code we used to handle Jam assertions.

Note that at the end of the function above we still need to base64-encode our signed XML and squash it into a single line.

Obtaining an Access Token

Armed with our assertion, we can now ask for an access token using a HTTP POST request against/oauth/token. The only thing worth mentioning here is that we need to pass a grant_typeparameter of urn:ietf:params:oauth:grant-type:saml2-bearer and include our OAuth client ID, our company ID and the assertion as well.

def get_access_token(self, assertion=None):
    if not assertion:
        assertion = self.get_local_assertion()

    token_request = dict(
    response = requests.post(self.access_token_url, data=token_request)
    token_data = response.json()
    return (token_data['access_token'], token_data['expires_in'])

Token Authentication for OData Requests

Having gotten an access token, we can now issue OData requests. All we need is to pass the token via the Authorization HTTP header:

headers["authorization"] = 'Bearer {}'.format(self.access_token)

A minor detail above that needs mentioning: the token needs to be prefixed with the Bearer string to indicate its type.

We can now pack everything together in a single SFSession class that wraps the "requests"get/post API and calls our SFSF server. Here is an example that fetches our user details:

SF_URL = os.getenv('SF_URL')
SF_SAML_PRIVATE_KEY = 'sf-private.pem'
SF_USER = sys.argv[1]

session = SFSession(server_url=SF_URL,
response = session.get("/odata/v2/User?$filter=userId eq '{}'".format(SF_USER))

The OData query is so simple, we don't even need to take care of much URL escapes most of the time. I really like that protocol.

Running the code above we get a JSON document:

{u'd': {u'results': [{u'__metadata': {u'type': u'SFOData.User',
                      u'addressLine1': u'1500 Fashion Island Blvd',
                      u'city': u'San Mateo',
                      u'country': u'United States',
                      u'custom10': u'admin',
                      u'dateOfCurrentPosition': u'/Date(983404800000)/',
                      u'dateOfPosition': u'/Date(1388534400000)/',
                      u'defaultLocale': u'en_US',
                      u'department': u'Industries (IND)',
                      u'division': u'Industries (IND)',
                      u'email': u'admin@ACEcompany.com',
                      u'firstName': u'Emily',
                      u'lastName': u'Clark',
                      u'zipCode': u'94404'}]}}

Source Code

The full source code is available on GitHub.

New files of interest in the sample project dir:

  • sf_saml.py generates and signs SAML assertions.
  • sap_sf.py authenticates and makes requests to the Jam API.
  • get_sf_user.py makes a sample API call that retrieves member details.

SAP Jam SAML Authentication Using Python

Author: Hristo Deshev

A quick introduction to authenticating to the SAP Jam API using SAML assertions.

We'll show how to generate and sign assertion XML documents, then use them to authenticate against the SAP Jam API using OAuth SAML bearer tokens. All that packaged in a small sample project with fully-working source code. read more ›

One of the most exciting things in our projects is working with new API's. This time it's the SAP Jam API. It has pretty good Java support, but I wanted to use it from our Python codebase, so I had to get creative.

Calling the SAP Jam API from Python is not too complex, but there are a few places that can get tricky. I managed to put the pieces together from several sources, so this is my attempt to document most if it:

  • Generating keys and registering them with SAP Jam.
  • Generating SAML assertion documents.
  • Signing the above as a SAML identity provider would.
  • Submitting an assertion to the server and getting back an OAuth SAML bearer token.
  • Authenticating API calls using the SAML bearer token.

Generating Keys

We are using 2048-bit RSA keys, generated with openssl:

# generate private key
openssl genrsa -out jam-private.pem 2048

# export public X509 certificate
openssl req -new -x509 -key jam-private.pem -out jam-public.cer -days 3650

Alternatively you can just run the generate_keys.sh script in my sample project (see below). The thing to note is that you generate keys once and keep the files. You will need them when registering your OAuth application and calling the API.

OAuth Access Configuration

We'll do this in the "Jam Admin" area. We need two pieces: an OAuth application and a trusted SAML identity provider.

OAuth Client Application

The OAuth Application is pretty straightforward. Register your domain and application URLs and do not set up an X509 certificate.

OAuth registration

Then we register a SAML Identity Provider (IdP). Note the IDP ID, Allowed Assertion Scope and X509 certificate fields:

SAML IdP configuration

SAML assertions

According to Wikipedia, the Security Assertion Markup Language (SAML) is an XML-based standard that lets different services handle authentication and authorization together. It is typically used to implement single sign-on (SSO) scenarios.

To use SAML with SAP Jam, you need to generate an assertion XML document describing the user you want to impersonate, yourself as the issuer, and some extra pieces of data such as validity periods. Now the full list:

  • Issuer. Typically a domain name such as example.com. You must use the one you provided when you registered the trusted IdP in the Jam admin area.
  • Subject. This is your user. SAML defines many ways to specify users, some allowing apps to use temporary opaque user ID's. We'll use the urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress name ID format and just pass the user email address.
  • Validity. We set up the proper SubjectConfirmationData, AuthStatement, and Conditions element with the correct authentication timestamp and and NotBefore and NotOnOrAfter points in time.
  • OAuth client ID. We pass our OAuth application's client ID.
  • Audience. Hardcoded to cubetree.com.

Here is how our full XML generation template looks like:

<Assertion xmlns="urn:oasis:names:tc:SAML:2.0:assertion"
xmlns:ns3="http://www.w3.org/2001/04/xmlenc#" ID="bo.ilic.test.idp"
IssueInstant="{issue_instant}" Version="2.0">
    <NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress">{user_id}</NameID>
    <SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer">

      <SubjectConfirmationData NotOnOrAfter="{not_valid_after}"
        Recipient="{jam_root_url}/api/v1/auth/token" />
  <Conditions NotBefore="{not_valid_before}"
  <AuthnStatement AuthnInstant="{auth_instant}"
    <Attribute Name="client_id">
      <AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema"
  <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
      <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315" />
      <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
      <Reference URI="">
          <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />

Signing Assertions

You might have already noticed the <Signature> element in the assertion document above, and most importantly its <SignatureValue> child node. This is, well, where our signature has to go.

Aside: Signing XML Documents

Signing plain text messages is easy - the message content has a single representation that can be used to compute the signature. Unfortunately, that is not the case with XML. For example, the two documents below look different, yet are completely equivalent:

<user name="John" email="john@example.com" />



To solve the multiple valid representations problem we need a way to normalize or canonicalize XML documents that will guarantee that, when applied to the two documents above, will yield the same results for both of them. That will make it possible for us to sign XML documents and verify signatures. Of course, people have come up with such algorithms already. For details, check Wikipedia's article on Canonical XML.

Signing XML in Python: xmlsec to the Rescue.

Implementing XML canonicalization isn't a simple task, but, well, this is Python and most of the time people have already solved problems like that before. We will be getting the xmlsec package off PyPI and using it to sign our assertions. It turns out to be quite easy, so I'll just give you the code:

def sign_assertion(xml_string, private_key):
    root = etree.fromstring(xml_string)

    signature_node = xmlsec.tree.find_node(root, xmlsec.Node.SIGNATURE)
    key = xmlsec.Key.from_file(private_key, xmlsec.KeyFormat.PEM)

    sign_context = xmlsec.SignatureContext()
    sign_context.key = key

    return etree.tostring(root)

Note that the code above assumes your XML string already contains a <Signature> node. Just like our XML template had above.

Obtaining SAML Bearer Tokens

Now that we have our assertion nicely signed, we need to pass it to the Jam server. We do that by base64-encoding the assertion document and getting rid of all whitespace, so that everything fits on a single line. We then issue a HTTP POST request to /api/v1/auth/token:

def request_token(self, assertion):
    encoded_assertion = re.sub(r'\s', '', assertion.encode('base64'))
    post_params = dict(

    token_url = self.url_for("/api/v1/auth/token")
    response = requests.post(token_url, data=post_params)
    return response.json()['access_token']

Note the saml2-bearer grant type above and the client_id and client_secret values. Again, you'll get the last two from your registered OAuth application settings in the Jam admin. Just have in mind that Jam calls the client_id value just key:

Jam OAuth client settings

Token Authentication

Once you've gotten hold of the token, you can issue API requests, by passing the token in an Authorization header. Here's how to do it:

headers["authorization"] = 'OAuth {}'.format(self.access_token)

Note the mandatory 'OAuth' prefix! And don't mind the lowercase "authorization" key -- HTTP headers are not case-sensitive.

Wrapping assertion generation, signing, and obtaining tokens in a simple JamSession class, we can now get our profile details, by issuing a HTTP GET request for /api/v1/members:

session = JamSession(server_url=JAM_URL,
response = session.get('/api/v1/members')

And here's the result we get back:

{u'assistant_ids': [],
 u'company-name': u'Ace',
 u'country_code': u'United States',
 u'created-at': 1378332130,
 u'current-status': {u'created-at': 1403259039,
                     u'id': 4452,
                     u'member-id': 98390,
                     u'source': u'Web',
                     u'status': u'<a href="dsasda">dsa</a>',
                     u'updated-at': 1403259039},
 u'direct_report_ids': [],
 u'email-addresses': [{u'address': u'admin@example.com',
                       u'location': u'Primary'}],
 u'first-name': u'Admin',
 u'handle': u'admin',
 u'id': 98390,

Full Source Code

I packaged the entire project and put it up on GitHub. Things of interest in there:

  • requirements.txt to set up your virtualenv.
  • generate_keys.sh to (duh!) generate RSA keys.
  • jam_saml.py generates and signs SAML assertions.
  • sap_jam.py authenticates and makes requests to the Jam API. You'll find the JamSession class you've seen above here.
  • get_jam_member.py makes a sample API call that retrieves member details.

Python and Django from dawn till dusk

Author: Emil Filipov

Want to learn Python and Django? Then this free seminar is just for you! Ninja training starts at the Telerik Academy in Sofia on Feb 3rd, 2013. read more ›

We've been invited to do another training session on Python and Django at the Telerik Academy. This time, it will be an intensive morning-to-evening seminar, with the aim of getting you from zero to hero on both Python and Django. Well, maybe not a true hero, but it will give you the basics of both technologies, so you can go on and study/work with them on your own. If you're in Sofia and getting into Python or Django has always been an unfulfilled childhood dream for you, or if you simply want to pick up some new and highly competitive skills for free, then waste not another minute - hurry to http://academy.telerik.com/seminars/python-and-django-development and reserve your seat!

Published in: Company News, Development

PyLogWatch is born

Author: Emil Filipov

Introducing PyLogWatch - simple and flexible Python utility allowing you to capture custom log files into the centralized Sentry logging server read more ›

Here, at MTR Design, we are managing multiple web apps, servers and system components. All of them generate some kind of logs. Most of the time the logs are trivial and contain nothing that we should be concerned about. There is the odd case, however, where some log gets an entry that truly deserves our attention. You see, the signal-to-noise ratio in most logs is very low, so going over all of the logs by hand is an extremely boring and time-consuming task. Yet, there may be "gems" inside the logs that you really want to act on ASAP - say, someone successfully breaking into your server, or email list going crazy and spamming your customers.

So, what solutions do we have at our disposal? The most noteworthy are Splunk (hosted service, expensive) and Logstash (Java, pain to install, maintain and customize). I did not like any of them. What I did like was Sentry, which has a logging client (called Raven) available in dozen languages. The only problem is that Sentry is meant for handling exceptions coming from applications - not for general purpose logging. 

Yet, Sentry has a lot of the features that we do need:

  • Centralized logging with nice Web UI
  • Users, permissions, projects
  • Aggregation, so that similar log messages get grouped together
  • Quick filters, letting you hide message classes you do not care about
  • Plugin system that lets you write your own message processing 
  • Flexible and easy to use logging clients

Since we already had Sentry for handling in-app logging, enabling it to handle general-purpose server logs felt like a very compelling idea. So we did it...

Enter PyLogWatch

... by writing a Python app that parses log files and feeds them to Sentry. The application is very small and simple, and you can run it on any server with a recent version of Python. You don't need to be root, there is no long-running daemon, and no special deployment considerations - just download, configure, run (by cron, or via other means of scheduling). Of course, PyLogWatch relies on you having a Sentry server, but that's not too hard to install either (see the docs), and you can always use the very affordable hosted Sentry service (see the pricing), which features a limited free account.

The PyLogWatch project is still in its infant stages - there are just a couple of *very* basic parsers (for Apache error logs and for syslog files), and no extensions for the Sentry server yet. Nevertheless, it has already proven very useful to us, since it enabled our developers to closely track the Apache error log files for the applications they "own", and swiftly react to any problem that shows up. In practice, each error line generates a "ticket" in Sentry, and it sticks up there until a project member explicitly marks it as resolved. As an optional feature, all project members receive an email whenever there is a new entry waiting to be resolved. 

What I love about this project is that it is a pretty much blank sheet of paper. I believe that using the combined power of custom parsers and Sentry plugins can yield magnificent results.

So what tool are you using for log tracking? What would do you like/dislike about it, and what would you ideally like it to do? Feel free to share your thoughts.

Another way to make a difference

Author: Emil Filipov

Here at MTR we try to make a difference every day, by challenging stereotypes and finding creative ways to deal with our tasks. This month, however, I will try to make a difference in another way - by doing some teaching. read more ›

Here at MTR we try to make a difference every day, by challenging stereotypes and finding creative ways to deal with our tasks. This month, however, I will try to make a difference in another way - by doing some teaching. A Django Crash Course (in Bulgarian) will take place on Aug 31st, in the headquarters of the initLab hackerspace in Sofia. I've been thinking about this for a while, since Django is basically unknown around here, and I finally found the time to do a little (pr|t)eaching. The plan is to cover the following topics:

1. Installing Python on Windows

2. Introduction to the Python interactive console and demonstrating basic Python constructs/syntax

3. Installing Django on Windows and playing with PYTHONPATH  + startproject

4. Installing Django on Linux; playing with runserver

5. Django Tutorial Part 1 

  • Folder structure
  • Running the development server
  • Database setup
  • Models/ORM
  • Playing with the models from the command line

6. Django Tutorial Part 2

  • Activating the Admin App
  • Adding our models to the Admin
  • Customizing the ModelAdmin

7. Django Tutorial Part 3

  • Routing addresses with the URL dispatcher
  • Writing views and rendering templates
  • Using template constructs
  • Named URLs and URL reversal in code/templates
  • Template resolution
  • Overriding Admin templates
  • Dealing with static media

8. Django Tutorial Part 4

  • Working with basic forms
  • Showcasing ModelForms
  • ModelForm security considerations

9. Making your life easy with Django Debug Toolbar

So there you have it - a Python fanboy trying to convince developers that they deserve better than PHP, during a 4-hour Django intro full of hate, love and ponies. The course is completely free, so do come by if you're in the mood for some webdev action!

Published in: Company News, Development
Tags: ,