I Guess We’re a Product Team After All

May 8th, 2009 38 Comments

I should order some vinyl stickers, paper doesn't wear well.Well, our big redesign of Connect, version 4.0, is finally live. Anthony and Rich are knocking out some final issues, but we cut over earlier this afternoon.

Luckily, it’s the weekend. So, our users aren’t using right now.

Unluckily, it’s the weekend, and everyone on the team has plans.

Next week, I’ll fill you in on the new stuff. So, tune in or out, depending on your interest level.

This is by far our biggest release, probably larger than when we switched to the Mix JRuby code line last Summer. Connect is just under two years old. Born in June 2007 as the IdeaFactory, we’ve managed to iteration four times, and the feature set is surprisingly large.

I’m continually impressed with Rich and Anthony’s ability to accomplish so much with so little.

As we realized this week, we’ve set the bar very high, and now that we have a lot of users, we have to move away from the startup mentality a bit and plan more.

For perspective, last June, when we switched to the JRuby code base, Connect had about 36,000 pageviews from 3,000 odd users. We’ve served 40,000 pageviews this week to just over 2,800 users, and Connect was down for about 12 hours while we tested the production migration.

Lately, a typical week would have topped the numbers for all of June 2008 and then some.

All these users have been busily creating data. I don’t have a solid number for the objects in Connect, but if I had to guess, I’d say more than 100,000. Anthony and Rich have completely rearchitected the data model in the new version, so that’s a lot of rows to touch.

Increased usage is both boon and bane for us because it means we have users doing work on Connect, which in turn means we can’t just mess around and bork it up without notifying them.

I’ve never been through a product testing feeling confident that I have a rock solid product. Testing is hard. I deal with it.

We went through about two weeks of beta-testing with the help of some excellent volunteer users and had the code pretty close to where we wanted it yesterday.

The plan was to let everyone know we were taking Connect down first thing today and to expect it to be live on Sunday morning.

I know, bummer, but we have more users overseas that domestically. So, we plan for India’s and Europe’s schedules.

Still, we thought this would minimize the downtime.

We were looking good, until I fired up a VM and decided to take a look in IE7.

Stop me if you’ve heard this one. Connect was unusable in IE7 and 8 due to all the AJAX we’ve introduced. As much as I’d like to skip coding for IE, we can’t. 26% of our users are on IE7, and we can’t abandon them.

So, we had an emergency huddle to decide what to do. Once again, Rich and Anthony rocked it out, and by the end of yesterday, we were ready to go on IE too.

I went to sleep preparing for a morning of testing.

I awoke to find that R&A had gone off script and taken Connect down early.

Then I get a call from a guy in EMEA. Uh oh.

Long story short, next time we need to do better.

Hard to believe that people here actually use Connect for real work, every day.

Seems like we’ve accomplished our mission, but along the way, we’ve become a product team, which is exactly what we started out to avoid. Plus, we’re not only a product team; because Connect is a web app, we’re also an operations outfit too, which is scary. I’ve carried a beeper. Do not want.

It’s not so bad. Planning won’t kill us, and it’ll be a while before we attempt a big release like this one. The operations piece will work itself out too. Frankly, Connect has very few outages (knock on wood). Not bad for a little JRuby app running on souped-up desktop boxes.

The tough part is going to be balancing the innovative stuff we want to build with the demands of our users–the classic product team dilemma.

We’ll get through the growing pains, and in the end, it’s all worth the effort.

We’ve managed to turn the ship a smidge, and I’m happy with our progress.

I have the coolest job at Oracle.

That’s my mantra for next week, when a full-scale, Facebook-style user revolt erupts.


Possibly Related Posts

  • http://only4left.jpiwowar.com jpiwowar

    Congratulating anyone on inadvertently becoming an operations and product team simultaneously would be mean, so I won't. ;-) I will, however, give sincere and hearty congrats for keeping a viral-to-vital product going at a pace of a new release every 6 months. That's flat-out awesome anywhere, and even more impressive given the size and geographic distribution of your team.

    And if you have users that care enough to launch a Facebook-style revolt, you have already won. ;-)

  • http://oraclenerd.com chet

    I'll echo jpiwowar, you have good problems.

    Also, a warning:

    “I have the coolest job at Oracle.”

    Others know this (like Billy Crime, I mean Cripe), and they're out for your job. :)

  • http://theappslab.com Jake

    Yay, I win!

    Thanks for the kind words, appreciated. We do what we can, but I think we'll have to slow down a bit to account for our users a little more in the future. It's going to be a challenge to balance cool new stuff with enhancements. A good problem to have though :)

    It's been quiet so far (knocking on wood), so maybe I'm being overly cautious.

  • http://theappslab.com Jake

    Billy would probably enjoy my job, but he wouldn't get to write books and travel around the country going to conferences :)

    Careful about wishing for things.

    Thanks for echoing John's kind words.

  • http://oraclenerd.com chet

    Can't you throw it over the wall like Oracle Mix?

  • http://theappslab.com Jake

    Heh, easier said than done my friend.

    Plus, we're not ready to do that just yet, although it may become a necessity if Connect gets too big or too mission-critical.

  • http://theappslab.com/ manalang

    Advice: Ditch Ubuntu proper and go with CrunchBang. It’s my new fav linux distro. It’s still Ubuntu under the covers but with better default choices of apps and a lot more lightweight.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • Gary

    I upgraded. I was on Hardy, but the promise of faster boot up time attracted me to Jaunty. I tried the ‘Upgrade’ button guessing that I might have to go through Intrepid before getting to Jaunty. Instead it decided that 99% of my packages had to be uninstalled and left me something that didn’t have bash, apt or anything usable or even fixable.
    So I did a fresh install of Jaunty. I think I’ve recovered most of my stuff, settings etc. I think it is booting up faster, but that may be wishful thinking or the fact it has been decluttered a bit.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://oraclenerd.com chet

    it would be fun if you could do that though…Hey, it's done. You take over now!

    I wish I had your problems. ;)

  • http://theappslab.com/ Jake

    Yeah, I forgot to plug your new favorite distro. Installing it on a VM. You have VPN working yet? Seems like it should.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Heh, see Rich’s comment re. CrunchBang, his new favorite distro.

    The bummer part is that Linux distros are still a lot work, but that’s the fun part too, if you have the time.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com Jake

    Sure, but refer to my comment about being careful what you wish to have.

  • http://theappslab.com/ manalang

    On VPN right now

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://www.andrejkoelewijn.com/ andrejk

    I upgraded with no problems, but had compiz turned off before upgrading: compiz doesn’t work well with mirrored screens or doing presentations on a beamer. Best new feature: jaunty seems to be a bit faster than 8.10.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://www.facebook.com/people/Graham-Hewett/1072965595 Graham Hewett

    I upgraded with no problems, compiz included. I also upgraded my mythtv box, the only issue I had there was going from Hardy to intrepid first where it overwrote my xorg.conf. Booting is much faster, which is a real boon on the myth box (my wife keeps shutting it down to save energy).

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Yeah, the boot time is definitely faster, he said recovering from yet another crash. I wonder if KVM is causing the problem.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Lots of upgrade success stories. Wishing I were one of them I really need to find time to start over from scratch, or to switch over to CrunchBang per Rich’s advice.

    It’s getting old.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://only4left.jpiwowar.com/ jpiwowar

    Whoa. Very keen. Two thoughts:

    1) Please tell me there’s a move afoot to get this into something customer-accessible like Beehive (I know, I know, Oracle employees not permitted to comment on product timelines, direction, features, etc. outside of official channels. Just consider it a +1 vote.)

    2) I can’t believe you missed a chance to nab an image like this one for this post.

    Nicely done.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://oraclenerd.com/ chet

    I say nothing because it looks like I’m stalking @jpiwowar (you should be watching hockey, not reading blogs).

    I have nothing new to add here.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://oraclenerd.com/ chet

    OK. I’ll say this.

    Well done.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • EspenBG

    I have one more success story; I couldn’t resist the upgrade option after having ignored it for a few days, and tried first at my old private Dell Inspiron 9200 with ATI. Did find any issues at all, just the smoothest upgrade I’ve ever seen (since my first SunOS upgrade test in 1998), so last week I did the same with the office Dell Inspiron 1720 (64-bit and Nvidia).

    Next and last is the desktop machine, the only one with an AMD processor. Things might still go wrong..

    I’m running Kubuntu, by the way.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://www.facebook.com/people/Frank-Bradley/586618505 Frank Bradley

    Just wanted to leave a comment here from an employee perspective. Good job guys, I really like the changes. I especially like the ability to bring aggregate feeds. Hoping this will help bring RSS to the masses.

    Thanks again for all the work.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Thanks John.

    Excellent call on the image. I will update and kudos for picking a CC-licensed one I can use.

    Rock on; oh, your +1 is duly noted

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Thank you good sir. Maybe next time I post, I’ll give you a heads up so you can be first.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Thanks for using Connect and driving its adoption. Without support from users like you, we don’t have a Connect.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Ack, you guys are killing me.

    I’m thinking my issue is either KVM or wireless related. When it freezes, all those blinky lights are flashing. I wish I had time to diagnose and fix it or downgrade.

    Not happening though.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://jeremymckenzie.com/ Jeremy A McKenzie

    +1 Would love to see it in something customer-accessible!

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Glad you like it.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://imterry.googlepages.com/ terrywang

    That’s weird. I guess it may be related to the Intel Integrated Graphic Driver. Someone posted workaround on Connect.

    I did the upgrade for my D620/D630 and Dell Precision Workstation in person, they all went smoothly. Faster boot, better sound solution (the D6X0 built-in mic works with Skye now), Compiz Fusion bugs were fixed.

    I am happy with Jaunty. To me the upgrade from Hardy to Intrepid was not pleasant. Too many bugs and degraded performance. So I switched back to Gentoo for a while. Now I am with Ubuntu again because of APT and PPA:-)

    U r right, Linux is an OS for its enthusiasts, DIY spirit required:-)

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • Mark Baker

    Just tried Posterous and am amazed and astounded that no one has thought of this before.

    The number of people I’ve heard say “I don’t know how to blog” is horrifying and trying to convince them that “it’s as easy as writing an email” has been very hard. Suddenly, it is as easy as writing an email!!

    If we can get that sort of functionality into Connect it would be stunning. I especially like that when I send a URL to something like Youtube it actually embeds the video in the entry. Very nice. If we could get that for email and tweet submissions, wow!!!

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Thanks for pointing me to the tips. I haven’t been monitoring that group, too busy fixing bugs, etc.

    Linux is definitely not for the faint-hearted, e.g. my Intrepid update went smoothly, yours did not. My Jaunty upgrade is a fail, yours was fine.

    Even so, I still think open source rules.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Our goal is to add more support for email, ultimately adding a Posterous-style integration.

    Email isn’t going away anytime soon, so we’re hoping to draw people into Connect via features like email this post, reply to comments via email, digest emails for groups, and blog by email.

    Glad you like the media plugin. Rich will be happy. It’s his shiny object.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://imterry.googlepages.com/ terrywang

    The new connect rocks. Good job;-)

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    Thanks, glad you like it.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://jauderho.com/ Jauder Ho

    At this point, I think I prefer using a Mac as a front end and sshing to a Linux box. Best of both worlds. Still on Hardy though since all the Linux boxes are servers.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • http://theappslab.com/ Jake

    I definitely prefer my Mac, but they gave me a Dell for work. So, I’ve been trying to make it work. Right now though, running a VM (VPN’ed into the network) on my Mac to avoid the random freezing issues is way better.

    It’s too bad b/c I really Ubuntu. Think I’ll have to bite the bullet and either downgrade to Intrepid or put CrunchBang on it.

    This comment was originally posted on http://theappslab.com/)”>Oracle AppsLab

  • Pingback: Do You Think Social Has Jumped the Shark? | Oracle