zomo technology

MySQL replication health

Tuesday, 09. 30. 2008  –  Category: sw

It’s possible for MySQL’s statement-based replication to get out of sync. I’ve yet to see a real example myself, perhaps because the DB abstraction layers I encounter not creating the “dangerous” queries that can make this happen. Still, other than monitoring replication health with SHOW SLAVE STATUS and friends I’ve sometimes been concerned about whether my MySQL slaves are faithful copies of the master. Finding that out during a failover moment is the wrong time.

The author of the piece linked above has scratched this itch with Maatkit, a suite of replication-focused MySQL tools which include some for checksumming tables.

This can run with standalone instances but most appealing is the way it runs across a replication chain: the statements to perform the table checksums are themselves introduced into the replication and store their results in a table alongside the master’s checksums. You can then compare and contrast on each slave: neat. All straightforward to roll into Nagios or other monitoring.

Firewalls

Friday, 09. 5. 2008  –  Category: vague

Ace quote from Bruce Schneier’s book Secrets and Lies:

The first firewalls were on trains. Coal-powered trains had a large furnace in the engine room, along with a pile of coal. The engineer would shovel coal into the engine. This process created coal dust, which was highly flammable. Occasionally the coal dust would catch fire, causing an engine fire that sometimes spread into the passenger cars. Since dead passengers reduced revenue, train engines were built with iron walls right behind the engine compartment. This stopped fires from spreading into the passenger cars, but didn’t protect the engineer between the coal pile and the furnace. (There’s a lesson for sysadmins in this somewhere.)

(My emphasis)

Using Single Sign-On To Integrate Ning With An External Site

Wednesday, 08. 20. 2008  –  Category: all, sw, web

Overview
Ning provide off the peg hosted social networks. The service is free unless you pay to not have their context-driven ads on your pages. Within a few minutes of sign-up you’re away.

Particulary cool is that they will let you at the source of your network. You can’t then wander off and run it elsewhere, it sits atop of their core web framework that you can’t see. With the source (which is encouragingly well written PHP) you can do perform all manner of modifications to bend the template network to your will.

Ning provide their own authentication system, and there’s no API to hook in someone else’s which is a hassle if you’re trying to build a social network alongside another site: maintaining a login for each site is going to annoy the users and be a nightmare to manage. Nil points.

One solution is to build a single sign-on system around what Ning already provide, which is robust, tested and better presented than anything we could achieve! I’d sketched out such a system for a proposal years ago but never had the opportunity to build it. A current project provided the perfect excuse to try this out.

Layout

Operation
The numbers here match those in the diagram.

1. User visits www.example.com (which for us happens to be a Rails) site.
2. The PageController notices the browser supplied no cookie and must therefore log on to Ning before proceeding. The site returns a redirect to Ning’s authentication page.
3. The browser follows this redirect to Ning.
4. Ning authenticates the user. The login code is modified from the original Ning behaviour. Here, it issues a redirect to sso.example.com with some parameters, including the user’s Ning ID and a salted hash to prevent spoofing. In this redirect Ning sets a range of cookies, including one that identifies the user to Ning.
5. The browser follows this redirect to the SSO server (which happens to be a Merb site – I wanted to try it out!)
6. The SSO app checks the provided hash against its own idea of what it should be. Assuming they match it considers the Ning ID to be valid. Finally, the SSO app issues another redirect along with cookies for just .example.com. This cookie identifies the user to the external site
7. In passing, the SSO app keeps track of users it has seen, and if this is a new user it will make an API request to Ning to fetch that user’s profile data and create a matching user on the www.example.com site.
8. The browser follows this last redirect back to Ning. It could be back to www.example.com or Ning depending on the situation.
9. Ning knows who the user is by merit of its cookies.
10. As does our external site.

Notes

• Having the SSO app different from the www.example.com site is perhaps a bit baroque. It works because the SSO app issues a cookie for .example.com which the browser will offer to both sso.example.com and www.example.com. In favour of this approach is that the SSO app is simple, and thus less likely to fail during development iterations than the nascent www.example.com site. A failure in the SSO app is bad, becaise it locks people out of Ning too. That Rails and Merb can share session data (and a database) is cool.
• The SSO app’s fetching of Ning profile data allows us to maintain a local version of a user’s profile to avoid the need to fetch it from Ning every time we need. There’s a nuance of Ning API authentication that meant I had to write a custom widget (Ning site component) to handle that.

Specifics
I was about to paste bits of modified Ning code, but I’ll need to check if I can under the various blurbs you agree too when signing up! Anyhow, the information above should help answer some of the requests for details from the Ning developers forum.