The Second Order Effect

• Oh, my, what a party. Can I turn 29 again? *glee* #
• @Madrox now I'm curious. #
• @heathervescent I hear you. I feel like a well intentioned foreign exchange student half the them. Thankfully people are often kind. in reply to heathervescent #
• Sorely needing is a process for organizing the various business ideas that come up. Need to prioritize, archive, develop, followup. Ideas? #
• @jory would love to know if you get any great advice. in reply to jory #
• All about the PyCon wireless network : http://www.tummy.com/Community/Articles/pycon2010-network/ #
• @gregvaneekhout I suspect it's not that we're stupider or crazier, but we're just being exposed to larger groups of people. Selection bias. in reply to gregvaneekhout #
• @brentbushnell Know if they're still interested in applicants — this looks *amazing* in reply to brentbushnell #
• I just bought the Mac Heist bundle. 7 Top Mac apps worth $260+ for only $20 and got 3 cool bonus apps free! http://bit.ly/heist-it #
• @gregvaneekhout I've never been there before – looks awesome! in reply to gregvaneekhout #
• I don't understand why 50-60 year olds insist upon telling me about the the state of the art in tech, while they're not aware of Linux. #

Powered by Twitter Tools

• Hmm, Suzanne Vega is in town right now, performing tonight at Chico.. #
• @MartianFrmPlnV wowie! For what! in reply to MartianFrmPlnV #
• I think this may be my favorite VHS tape ever. http://twitpic.com/15j00m #
• @Madrox you made my day. in reply to Madrox #
• @andysternberg because folks are afraid of bad education, being homeless or unemployed. Not that I agree… in reply to andysternberg #
• @mikeprasad neat idea! in reply to mikeprasad #

Powered by Twitter Tools

• Just called sir at the airport parking lot – by the sort of person who calls any stranger sir (I'm one of them). Disturbing nonetheless. #
• @randbot congrats! in reply to randbot #
• @boogah I love that area. Go by Cambria next time, I know a little inn there to die for. in reply to boogah #
• @boogah @egradman I've used Dreamhost PS and I was decently impressed. in reply to boogah #
• Leaving for San Jose for the Billy Joel/Elton John concert tonight. Bonus : Going with one of my dearest friends from undergrad. #
• @boogah off the top of my head, think it's Little Sur Inn. Do visit Cambria on this trip- eat at Robin's or Wild Ginger. Have fun! in reply to boogah #
• @boogah If I didn't have 20 of my own servers, I'd be using PS more. That link is neat! in reply to boogah #
• Elton John and Billy Joel – "Dont let the sun go down" http://twitpic.com/13pik5 #
• @rebelprince Totally random, but I'm with Tara right now! in reply to rebelprince #
• "I'm Billy Joel's dad. Billy was sick today, but I know all of his shit. You've got to be saying, man, he's old!" – Billy Joel #
• @lifeofreilly I am. in reply to lifeofreilly #
• @HilaryAndrew teehee in reply to HilaryAndrew #
• RT @codinghorror: "all email responses regardless of recipient or subject will be five sentences or less" http://five.sentenc.es – hmm #
• @gregvaneekhout are there any pirates or monkeys near by? Or better yet – Monkees? in reply to gregvaneekhout #
• @stevenswei Andrew used to use Amazon Mechanical Turk to do translations. It went … okay. in reply to stevenswei #
• Rate of increase :
  Opportunities:exponential.
  $/tech resources:quadratic.
  Amount of things I can do simultaneously:linear.
  This is cruel. #
• @timiat Pink Martini "hang on little tomato". Billy Joel "The Stranger" in reply to timiat #
• Mai Tai's, Blue Hawaiian's and Hawaiian Punch. But what drink to make for the sugar/rum overloaded? Margaritas? Whiskey Sours? #
• @lifeofreilly @HilaryAndrew @Colleenky @gregvaneekhout @dustchick @keltraine – Jonathan Coulton at the Largo! Whoo hoo! in reply to lifeofreilly #
• @lifeofreilly grin. They are some hot geek women. in reply to lifeofreilly #

Powered by Twitter Tools

I created a repository at Geekymedia RPMs that contains :

Bashton’s EL5 Python 2.5
My EL5 Python 2.6
Chris Lea’s EL5 Python 2.62 (http://yum.chrislea.com/centos/5/SRPMS/)

Projects to do:
Python 2.64
Asterisk 1.62 for AsteriskNOW!

Download this repo file and place in /etc/yum.repos.d.

Enjoy.

• I decided to stop working 1.8 hrs ago – I'm just now pulling myself away and having a meal before the clubs. I have no discipline. #
• @RichardABecker I haven't had an eight hour workday since I was 14. I'm told it's doable, but first step is to lose the home office. in reply to RichardABecker #
• @Colleenky mmmm cheese in reply to Colleenky #
• I have never regretted going out 80s clubbing, ever, in my life. I have to get Enigma out to Blue or Clockwork soon – it's high time. #
• @dustchick what's so terrible about French, save for it's lack of relevance? in reply to dustchick #
• @cyuskoff where? in reply to cyuskoff #
• Damned openwsman update uses /dev/random rather than urandom in cert generation. In a boot script. Drove to server to fix after 2hr wait. #
• @andysternberg jazz in LA? in reply to andysternberg #
• @stevenswei grin. Using other languages since Py makes me sad. in reply to stevenswei #
• @timiat what should they have taken out instead? in reply to timiat #
• @kjkjerstin I lost mine in Hawaii. (Pilot Knight) Bought a new one the day I got home. I know your pain. in reply to kjkjerstin #
• @andysternberg I assumed I (and you) would be at a BIL event. in reply to andysternberg #
• @timiat pretty good way of influencing basic users to upgrade without taking away real functionality. Sounds saavy (and evil) to me. in reply to timiat #
• While it definitely took away from my plans for today, that last hour debugging/fixing that SQL Server locking issue felt really good. #
• @andysternberg Sad I didn't make it – sounds like you enjoyed yourself though. in reply to andysternberg #
• @mikeprasad are you at the food cart festival? in reply to mikeprasad #
• @Wishme man, now I want one too. in reply to Wishme #
• @Colleenky turns out the Goodwill on Santa Monica does ewaste recycling for free. in reply to Colleenky #
• @stevenswei did you ever end up reading their ebook? in reply to stevenswei #
• @chews did you go to PS or LB this year? I think I missed the window to apply for next year in reply to chews #
• @Wishme we're going late March – it can't come soon enough. in reply to Wishme #
• Now to find a Viking Costume for tonight. #
• Man, a Gil Elvgren pop-up book. Pin-ups for the five year old crowd. Egads. #
• @andysternberg *why am I not surprised you are at lucent? in reply to andysternberg #
• RT @ThatKevinSmith: "No threesome with the Missus then?" On Valentine's Day? Please, sir – we're in love. The threesome's the night before. #
• In the fantastic kitchen of a notable director. Funny thing to realize – "I actually prefer mine to this one." except for hanging pot rack #
• @dustchick ask and ye shall recieve. http://twitpic.com/136tax in reply to dustchick #
• Although I wouldn't mind having an indoor hottub. #

Powered by Twitter Tools

Tasks keeping me away from what I’d rather be doing this past few days :

Did some updates on tardis, our general purpose linux server (all the servers have names from Dr. Who) – and updated the Dell OpenManage tools. Reboot. Simple, right?

The machine responded to pings, and nothing else.

It was 2pm. Traffic in LA becomes a nightmare somewhere around 3:30pm. Jumped in the car to crash cart the server (because the tech on site didn’t know what a crash cart was and when I described it – said ‘that sounds complicated.’ Not happy with our colo right now.)

Arrived to find the server paused — not hung, just waiting, on a “Starting openwsmand…” message. Turns out, there’s a new service in the latest version of OpenManage that requires a new OpenSSL certificate for functioning. This, on it’s own, is okay – but two problems :

1. The cert is generated by the boottime init.d script if it’s missing.
2. The cert is generated using /dev/random as the entropy source

The latter is a security concern, but should never be paired with the top. I’ve had servers hang for hours waiting for /dev/random to generate enough entropy.

This was easily enough fixed, but the stress of driving like a madman, (and back) to make it before traffic shut down all routes to West LA was something I could do without. I’m going to patch against the project and see if they’ll fix this ridiculous behavior, and I’ve started looking into a remote KVM solution like kvm2ethernet – just call the colo and ask them to plug into a particular server. Thanks to this post for cutting the debugging time massively.

The other issue was that customers weren’t able to sign up for bits of today because there was a lock residing on one of our DB tables. We purge our Database (about 50G) monthly, but the cruft of leftover billing records take up huge amounts of space and deleting them can be a problem – long table scans, and Microsoft SQL Server does a table lock – and that’s the ball game. The credit card server can’t track that a valid charge was placed, so it terminates instead. We started with a query from our Marketer/Data Analysis guy looking like :

1
2
3
4

DELETE FROM billing WHERE 
			AND start_date_time >= '18-JUL-2009' 
			AND start_date_time < '24-JUL-2009'
			AND node_type NOT IN (3,4,5)

(He wanted to delete up until 30-AUG, but was slicing it up in the hopes of avoiding this problem.)

The above is about 300,000 rows.

A few issues, however :

1. billing has a clustered index around account_id. This makes perfect sense, the data is almost always referenced with respect to a particular customer, and ensuring that those records are all adjacent to each other on storage is common sense. However, the above query would be running all around the 9Gb table removing rows.
2. billing doesn’t have an index on node_type, meaning each row has to be fetched before it can be selected for deletion.

The latter is less of a problem if the select and the delete were separated, but the table lock existed throughout the query.

In looking to solve it, one approach was to force MSSQL to use ROWLOCK (and disable escalation from ROWLOCK to TABLELOCK), but this was going to be a performance hit. I considered trying NOLOCK, but I wasn’t sure of what the ramifications would be, and I really didn’t want to spend hours fixing a crash database/corrupted data.

The final solution was to carve up the deletes into more manageable bits. SQL Server Interactive can be set to execute on a limited number of rows – so we could delete, say, 1000 rows, pause (let other things have access to the table), and then continue. And, now that we know about the clustering, why make the DELETE run across the entire table – why not allow it to trim one section at a time? And thusly we have…
(I’m not really familiar with SQL Server Syntax, so this is a first effort. It’s lousy code, but a decent query)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

DECLARE @ACCOUNTSTEP int
DECLARE @WINDOWMIN int
DECLARE @WINDOWMAX int
DECLARE @ACCOUNTMAX int
DECLARE @WINDOWCOUNT int
DECLARE @STEPS int
 
SET @STEPS = 20
 
SET @ACCOUNTMAX = (SELECT max(account_id) FROM accounts)
SET @ACCOUNTSTEP = (ceiling(@ACCOUNTMAX * 1/@STEPS))
SET @WINDOWMAX=0
SET @WINDOWCOUNT=0
PRINT CAST(@STEPS AS CHAR(10)) + '+1 Steps of Size ' + CAST(@ACCOUNTSTEP AS CHAR(10)) + ' leading up to ' + CAST(@ACCOUNTMAX AS CHAR(10))
SET ROWCOUNT 2000
 
moreaccounts:
	SET @WINDOWCOUNT = @WINDOWCOUNT + 1
	SET @WINDOWMIN = @WINDOWMAX
	SET @WINDOWMAX = @ACCOUNTSTEP * @WINDOWCOUNT
	IF @WINDOWMIN > @ACCOUNTMAX GOTO done
	PRINT 'Now processing accounts between ' + CAST(@WINDOWMIN AS char(10)) + ' and ' + CAST(@WINDOWMAX AS char(10))
	deletemore:	
		PRINT 'Deleting 2000 rows.'
		waitfor delay '0:0:01'
		DELETE FROM billing WHERE account_id >= @WINDOWMIN AND account_id < @WINDOWMAX 
			AND start_date_time >= '18-JUL-2009' 
			AND start_date_time < '30-AUG-2009'
			AND node_type NOT IN (3,4,5)
		IF @@ROWCOUNT > 0 GOTO deletemore
	PRINT 'Done with this set. Sleeping.'
	waitfor delay '0:0:03'
	GOTO moreaccounts
done:

Which effectively breaks out to :

• Get the total number of accounts
• For every twentieth window of accounts
Delete 2000 rows. Wait a second. Repeat until all rows gone.
• Wait three seconds.

Ran the query, no locking issues at all, Table purged in about an hour. Victory!

I wanted to get Zabbix monitoring working for our Asterisk boxen – I went a bit overboard, and resulted in zasterisk.

I whipped a script together that logs into the Asterisk AMI and returns some key Asterisk data for Zabbix, my favorite server management tool:

Skype (Licenses,Calls)
G729 (Licenses, Encoders,Decoders)
Asterisk (Calls, Channels)

This is all without the yucky launching of the asterisk CLI or dealing with sudoers. There’s a semaphore lock in place (making this Linux/UNIX only for now) to prevent the AMI from being accessed by more than one check at a time.

Details are available here : Google Code for Zasterisk

As always, would love feedback, bug reports, patches, or suggestions for new features. It’s *very* raw, I’m just getting it out there to have it out there, but I know there’s stuff to clean up/fix. It’s missing an XML import file as well.

I’ve been having a growing love affair with Python for over a year, and now that I’ve got a few projects that I’m working on – I’m using Python as my primary language.

I had a happy moment when refactoring a section of annoying if/then/else sections …

if object.type == "Vendor1":
    foo = Vendor1()
elif object.type == "Vendor2":
    foo = Vendor2()
elif object.type == "Vendor3":
    foo = Vendor3()
else:
    raise Exception("Not a handled type")

Python is a functional language, and this means (among other things) that functions, classes, and all sorts of stuff can exist in dictionaries. So, instead, I can do

lookup = { "Vendor1" => Vendor1, "Vendor2" => Vendor2, "Vendor3" => Vendor3}
try:
    foo = lookup[object.type]()
except KeyError:
    raise Exception("Not a handled type")

It’s not a big thing, but it made me happy — and the example can be extended …

lookup = { 
    "Vendor1" => { "url" => blah, "threads" =>"blah", "obj" => Vendor1() }
}
 
lookup["Vendor1"]["url"]
#equals blah

Today, it’s all about the little things.

So, I’ve got a Dell Dimension 8400. One nice tidbit that they included was a front headphone plug that actually interrupts the speaker output; not a difficult thing to do, granted, but its nice. One thing that surprised me was that the Windows Startup sound WOULD be played out the main speakers no matter what — making me realize that the interrupt was a software function, not a hardware switch.

Until it stopped working last week. Hours of time w/ Dell Support (which, by the way, seems to be outsourced to New Delhi and not a single person fully understood my problem), Dell forums and Google gave me little. The deal is, is that the Audigys normally come with a front panel with plugs; which isn’t available in the Dell world. And yet, it still uses that internal routing to handle the headphone plug.

So, has your headphone switch in stopped working? Reinstalling the Dell drivers does nothing. Google suggests that you turn on the setting in the AudioHQ control panel — but it isn’t there.

Installing Creative’s stock drivers over your Dell drivers just gives me the shivers. Oh, and it doesn’t work. All I ended up doing was finding a driver package, say, for instance,SBA2_PCWDRV_LB_1_84_55-R1. Extract the archive (run it and look in your temp directory while its running; or even better, just use WinRAR to extract it), and go to Drivers\WDM\Common\ and run CTPanel.exe. (Or just download my copy).

Run it, go to device settings, and voila! “Mute speakers upon Headphone Jack Detection.”

That’s it!

I used to host a large archive of ‘This American Life’ in MP3 format before they had it available on their website as a podcast.

I was contacted, quite politely, and asked to remove the files from access, and I complied.

Go to TAL and stream down the eps or use the podcast.