What methods and objects would a typical transaction use?

Developer
Jan 12, 2009 at 6:51 PM
This wrapper for the Netflix API looks pretty broad and comprehensive - seems like the methods and properties that are available cover a number of steps in complex Netflix transactions.

I haven't studied the raw Netflix API or documentation, and ideally (and in the long term), I would like to be able to simply use this wrapper, and avoid having to also dig into the underlying API that it wraps.  However, at the moment it's not self-evident to me which methods and properties are needed when trying to complete an end-to-end typical transaction, and in what order they are intended to be called.

John, based on your design intent of this wrapper, which methods and/or properties would you expect to use to complete a typical transaction?  e.g. what methods/properties/classes would you call (and in what order) to add a new Title to a user's Queue?
Coordinator
Jan 16, 2009 at 6:36 AM
Well the first thing every program needs is the request and access tokens. You can see how to use them from the test driver.

From there, it is hard to say. Most of the properties are informational, you only call them if you actually want to show that information on the screen. Try thinking about it the other way. Tell me what you want to do, and I'll tell you the steps to do it.
Developer
Jan 26, 2009 at 1:02 AM
John, thanks for pointing out the TestDriver code - that certainly gets me a few steps further.

However, I must admit that after a few hours of trying to build up my own code from scratch, and trying to sort out which pieces of the TestDriver code are meant for what, I'm making very little progress for the time I've invested.  Not your fault - I'm sure it's because I don't know enough about coding - but I'm hoping you can help set me aright.

I'm focused on Module1.vb for now - hopefully that's the class file with which you *intended* other consumers of this code to start:
  • It appears there are at least five separate members of My.Settings - ConsumerKey, ConsumerSecret, AccessToken, AccessTokenSecret, UserId
  • I've looked at this code a few times, and without any inline comments I'm stumped as to which of these Settings values:
    • needs to be distributed with the application vs. those that are acquired at runtime?
    • are static and which are dynamic?
    • are needed to acquire a subsequently-acquired value?
  • I'm especially confused by your TryRead() calls up front - I don't understand why, but it looks to me like you're obtaining the values of ConsumerKey and ConsumerSecret from the Console
    • Is this something you'd expect a user to know off the top of their head - or know where to find?
  • Then I notice that the UserId setting must already be present in the app's Settings file, 'cause I can't see a way for it to be enumerated from the system or obtained from the user
I completely understand if this is code you whipped together to make it easier for yourself to debug things as you went along.  However, I guess what I'm finding (and what I hope these questions help illustrate) is that the assertion that we should be able to follow the TestDriver code to make use of the NetflixWrapper assembly is a mite harder than it sounds.

John, I want to emphasize - I don't intend this to be complaining or criticism.  I sincerely believe you have the best intentions, and I hope that my requests can help in some small way to improve the usability of the hard work you've put into this project.  (I really hope this isn't just irritating.)
Mar 9, 2009 at 12:03 AM
>> Tell me what you want to do, and I'll tell you the steps to do it.

OK, I have a Netflix ID.  70078600 which is MythBusters: Collection 2

how do I get a list of episodes? 
Thanks!

PS So far this looks awesome! 
Developer
Aug 5, 2009 at 5:10 PM
Edited Aug 5, 2009 at 5:11 PM

This is a follow-up to my previous questions, to try to document what I've learned from my very sporadic/infrequent attempts at coding up my Ratings application (and highlight what still isn't clear):

  • ConsumerKey and ConsumerSecret are the Application Key and Shared Secret that Netflix issues to each API developer.  This is one pair of data that would be identical across all installs of a particular application.  I've pasted those values into the Application.Properties.Settings values in my application, and I don't mess with them ever again.
  • My NetflixRatings assembly starts off by initializing the "user" object, which is to say get all this OAuth authentication, token trading, application linking out of the way
  • Leveraging the WrapNetflix assembly, my code:
    1. instantiates a new NetflixConnection object (passing in the ConsumerKey and ConsumerSecret values)
    2. checks the NetflixAccessToken to see if it's empty (i.e. if the app has already authenticated with Netflix or not)
    3. calls GenerateRequestToken()
    4. calls Process.Start, passing in the PermissionUrl associated with the Request Token object
    5. calls ConvertToAccessToken() on the Request Token object
    6. saves the Access Token's UserId, Token and TokenSecret back to the application settings
    7. finally, instantiates a new User object (passing in the Access Token and NetflixConnection objects)

However, at the moment the sequence (1-7) throws a NetflixException at some point, spitting back the following error from the server:

<?xml version="1.0" standalone="yes"?>
    <status>
        <status_code>401</status_code>
        <message>Expired or invalid request token</message>
    </status>

(I notice as well that my default browser has thrown up the "SharpRatings wants to access your Netflix Account" page - whose URL looks like https://api-user.netflix.com/oauth/login?application_name=SharpRatings&login_url=https%3A%2F%2Fapi-user.netflix.com%2Foauth%2Flogin%3Foauth_token%3D[oauth_token]&oauth_token=[oauth_token]&oauth_consumer_key=[key]&oauth_timestamp=1249447289&oauth_nonce=[nonce]&oauth_signature_method=HMAC-SHA1&oauth_signature=[oauth_signature]%3D  - of course, I've obfuscated the unique values generated by my application with the [values].)

I'm going to do some work to isolate exactly which line of code is responsible for the 401/expired/invalid error, but at the root of it all, I'm concerned that (a) this is not a good user experience for first-time users of such an application, and should be prevented, and (b) it's probably just a one-time occurrence for each developer until their application has cached its authentication from the Netflix server - so it probably doesn't stay on the radar of too many developers.

If anyone has a suggested way of addressing this in code, I'd be rather grateful to not end up stumped for another couple of months. :)

P.S. Here's the core C# code that corresponds to  (1-7) above:

 

netflixConnection = new NetflixConnection(Properties.Settings.Default.ConsumerKey, Properties.Settings.Default.ConsumerSecret);

if (String.IsNullOrEmpty(NetflixAccessToken))
{
    RequestToken currentRequestToken = netflixConnection.GenerateRequestToken();
    Process.Start(currentRequestToken.PermissionUrl);
    userAccessToken = currentRequestToken.ConvertToAccessToken();

    Properties.Settings.Default.UserId = userAccessToken.UserId;
    Properties.Settings.Default.UserAccessToken = userAccessToken.Token;
    Properties.Settings.Default.UserAccessTokenSecret = userAccessToken.TokenSecret;
    Properties.Settings.Default.Save();
}

user = new User(userAccessToken, netflixConnection);

 

Developer
Aug 5, 2009 at 8:44 PM

Update: through some stepwise debugging, I was able to identify the specific line of code that's responsible for this error, and I think I understand the root cause (but not how to solve it):

  • the call to ConvertToAccessToken() is what immediately precedes the NetflixException
  • In my debugging I was exposed to the XML comments for the ConvertToAccessToken() call, which says "Do not call this method until after the user approves your application. Calling it prematurely will will foul the Request Token."
  • There is currently no "pausing" functionality implemented in my code (nor, from what I gather, in the wrapNetflix code), so my code
    • executes the Process.Start() call to present the user with the "SharpRatings wants to access your Netflix Account" browser window
    • then immediately calls ConvertToAccessToken() to enable the app's authenticated access to the user's Netflix account
  • Without a "wait for the User to complete the browser request" step, the ConvertToAccessToken() call will always fail [at least until the access request is granted through one out of band means or another]

I can think of a few ways in theory to handle this, but I don't have any practical experience in handling this scenario in code, so I feel like I'm stuck without some tips, pointers or assistance from others who've also faced this problem:

  1. Intercept the NetflixException, and if a "401" error is received, then jump out and loop back into the initialization code until no "401" error is received.  [Well, with something there to ensure the user isn't exposed to an infinite loop.]
  2. Call some .NET function that wraps around the Process.Start() call, and waits for the browser to be closed before proceeding.  [This would likely have to introduce multi-threading into the mix, which I've never done, to ensure that the Main() procedure doesn't get blocked - and thus hang the UI part of the app.]
  3. Implement some lightweight browser functionality inside the app, to present the access request page "inline" in the apps' UI.  [This can get a bit hairy with all the security challenges around implementing a browser, and all the potential for anti-malware apps to intercept and block this behaviour.  It's also a bit risky to screw with the look and feel of a user's normal web browsing experience - a request like this in an unfamiliar window might look like the very hacking attempt that the 'access requested' page warns the user to deny - and thus could lead to users considering this spyware - or thinking it's been infected with spyware.]

Does anyone know of an open-source application that handles these kinds of "third-party authorization" requests in a reasonably functional/elegant way?  (Whether they're for Netflix, any other oAuth-based service, or anything else like Google, Facebook or the rest]  I'd be happy to follow any leads people might have to offer - or any sample code they have that does the trick.  Thanks all!

Developer
Aug 6, 2009 at 5:52 PM

Another in the continuing series of Mike talking to himself... :)

I went looking for open-source, desktop applications that implement the oAuth protocol, hoping that they'd have similar challenges to overcome.  Turns out that at least in the case of the first good desktop app I found, they're dealing with the exact same issue!

LINQ to Twitter implements oAuth authentication to Twitter from a demo desktop app (via its own Twitter/oAuth code), including this little tidbit in the DesktopOAuthAuthorization.cs module (note the comment stating "...and blocks until it is complete" - very exciting, but not entirely understood on my end):

/// <summary>
/// Invokes the entire authorization flow and blocks until it is complete.
/// </summary>
/// <returns>
/// The extra data included in the last OAuth leg from Twitter that contains the user id and screen name.
/// </returns>
/// <remarks>The <see cref="GetVerifier"/> property MUST be set before this call.</remarks>
public override IDictionary<string, string> Authorize()
{
if (this.GetVerifier == null)
{
throw new InvalidOperationException(string.Format(CultureInfo.CurrentCulture, "{0}.GetVerifier must be set first.", this.GetType().Name));
}

Process.Start(this.BeginAuthorize().AbsoluteUri);
return this.CompleteAuthorize();
}
Developer
Aug 7, 2009 at 8:25 PM

... and thus did Mike finally understand not one, but *two* ways to tackle this task:

  1. I dropped into the LINQ to Twitter discussions and got the dirt straight from one of their developers.  It looks like they're taking advantage of something called "callback delegates" in the DotNetOpenAuth assembly (which the LINQ to Twitter project have converted over to using).
  2. JoeMayo's July 1st post to an older thread in L2T discussions outlines an approach to use the Console to crudely "block" the thread from completing - by asking for input from the user to signal to the app that the authorization has been completed.

For me, I'd love to see the DotNetOpenAuth assembly integrated into the wrapNetflix project, so that we can inherit all the great work they've done.  (The tidbits I've seen in the L2T discussions seem to indicate that the DNOA assembly is a high-quality, robust piece of code that covered all the needs L2T had).

I don't know if I could go it alone (nor would I want this project to depend on me for that, in case I moved slower than necessary), so I'd like to hear if anyone else is interested in working with me on a possible integration?  John, have you had any thoughts on what it would take to switch the wrapNetflix code to DNOA, or whether it'd be worth the trouble?

Coordinator
Sep 11, 2009 at 8:34 AM
Man, trying to keep on top of my email seems downright impossible.
 
If you really think DNOA is worth the effort then by all means go for it. I'm not going to be overly protective of this code, I only started this project because I wanted a way to randomize my queue. Just make sure it works in the DEV branch before you merge it into MAIN and release a build.
 
Jonathan

On Fri, Aug 7, 2009 at 1:25 PM, MIkesl <notifications@codeplex.com> wrote:

From: MIkesl

... and thus did Mike finally understand not one, but *two* ways to tackle this task:

  1. I dropped into the LINQ to Twitter discussions and got the dirt straight from one of their developers.  It looks like they're taking advantage of something called "callback delegates" in the DotNetOpenAuth assembly (which the LINQ to Twitter project have converted over to using).
  2. JoeMayo's July 1st post to an older thread in L2T discussions outlines an approach to use the Console to crudely "block" the thread from completing - by asking for input from the user to signal to the app that the authorization has been completed.

For me, I'd love to see the DotNetOpenAuth assembly integrated into the wrapNetflix project, so that we can inherit all the great work they've done.  (The tidbits I've seen in the L2T discussions seem to indicate that the DNOA assembly is a high-quality, robust piece of code that covered all the needs L2T had).

I don't know if I could go it alone (nor would I want this project to depend on me for that, in case I moved slower than necessary), so I'd like to hear if anyone else is interested in working with me on a possible integration?  John, have you had any thoughts on what it would take to switch the wrapNetflix code to DNOA, or whether it'd be worth the trouble?

Read the full discussion online.

To add a post to this discussion, reply to this email (WrapNetflix@discussions.codeplex.com)

To start a new discussion for this project, email WrapNetflix@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Mar 9, 2011 at 4:21 AM

If there is anyone out there still wondering how to gain user approved account access, the steps are as follows:

1) respond to some UI action that requires logging in to netflix
2) in that handler, make these calls:
        NetflixConnection nfCon = new NetflixConnection(strKey, strSharedSecret);
        RequestToken reqTok = nfCon.GenerateRequestToken();
        string strUrl = reqTok.PermissionUrl + "&oauth_callback=" + BaseSiteUrl + "Default.aspx";
        
        // save some data as cookies, see SaveSetting implementation below
        // RequestToken.GetRawData is a new routine that needs to be added to the
        // WrapNetflix source, see step 4 below
        SaveSetting("NetflixRawData", reqTok.GetRawData());
        SaveSetting("NetflixPermissionUrl", strUrl);
        
        Response.Redirect(strUrl);

    SaveSetting is defined as follows:
        public static void SaveSetting(string strName, string strValue)
        {
            HttpContext hc = HttpContext.Current;
            hc.Response.Cookies[strName].Value = strValue;
        }
    
    BaseSiteUrl is defined as:
        public static string BaseSiteUrl
        {
            get
            {
                HttpContext context = HttpContext.Current;
                string baseUrl = context.Request.Url.Scheme + "://" + context.Request.Url.Authority + context.Request.ApplicationPath.TrimEnd('/') + '/';
                return baseUrl;
            }
        }
        
3) in your Page_Load routine for the page you identified in the oauth_callback parameter of the querystring (Default.aspx above) do this:
        string str_oauth_token = Request.QueryString["oauth_token"];
        if (!string.IsNullOrEmpty(str_oauth_token))
        {
            // if the parameter exists you can be more than reasonably sure the user is being redirected back from Netflix
            NetflixConnection nfCon = new NetflixConnection(strKey, strSharedSecret);
            // get the previously saved settings
            // this is our poor man's object persistence
            // we need to get the RequestToken and NetflixConnection back to the same state they were 
            // when we first created them in step 2 above
            string strRawData = GetSetting("NetflixRawData");
            string strPermissionUrl = GetSetting("NetflixPermissionUrl");
            // the following is a new c-tor you need to add to the WrapNetflix code, see step 4 below.
            RequestToken reqTok = new RequestToken(nfCon, strRawData, strPermissionUrl); 
            AccessToken accTok = reqTok.ConvertToAccessToken();
            User user = new User(accTok, nfCon);            
            // At this point you should have full access to the users Netflix account. 
        }
        
    GetSetting is defined as follows:
        public static string GetSetting(string strName)
        {
            HttpContext hc = HttpContext.Current;

            if (hc.Request.Cookies[strName] != null)
                return hc.Request.Cookies[strName].Value;

            return string.Empty;
        }
        
4) Here's where it gets a little ugly. Add these two new routines to WrapNetflix/UserResources/RequestToken.vb in the WrapNeflix source. 

    Public Sub New(ByVal connection As NetflixConnection, ByVal rawRequestToken As String, ByVal PermissionUrl As String)
        Dim parsedData As New ClrExtensions.Net.Rest.QueryParameterCollection(rawRequestToken)
        m_Connection = connection
        m_RawData = rawRequestToken
        m_Token = parsedData("oauth_token")
        m_TokenSecret = parsedData("oauth_token_secret")
        m_PermissionUrl = PermissionUrl
    End Sub

    Public Function GetRawData() As String
        GetRawData = m_RawData
    End Function

I think that is all of the changes. I've successfully run this code and have gotten back a legit User instance (see step 3 above). However,
none of this should be necessary. You should be able to instantiate an AccessToken from the returned oauth_token (when Netflix
redirects back), but I was unable to get that to work, as the constructor is expecting more than just the token to create the object. 
It creates the object, but throws an exception when you try to use it. I notice that the Java wrapper seems to imply that it can proceed with 
just this info, and so I'm thinking about getting that source code to see if I can figure out how they're doing it. If that's successfull, I'll 
follow up with that info.   
        
If you need Netflix movie info that doesn't pertain to a particular user, your task is much easier:
    NetflixConnection nfCon = new NetflixConnection(strKey, strSharedSecret);
    CatalogTitleCollection ctc = nfCon.Catalog.TitleSearch("Revolver", 5);
        
And that's it, just those two lines of code and you're getting data back from Netflix.