SeanColombo.com

After upgrading to the latest version of WordPress, the text of all of the categories were deleted. It appears this may be my fault for not disabling all of the plugins before doing the upgrade (oops). They’re back now though. It was somewhat annoying to have to do, but as far as “data loss error”s go, it wasn’t very bad at all.

I noticed a bunch of other people online had similar problems but nobody mentioned a solution so I figured I might as well throw up a quick description of what I did.

I used google to find a cached version of my page which listed all of the categories in the side-bar. Now I had all of the categories but I didn’t know which id in the database matched which category. I found that out by running this query:
SELECT post_title, term_taxonomy_id FROM wp_posts,wp_term_relationships WHERE wp_term_relationships.object_id=wp_posts.ID ORDER BY term_taxonomy_id;
Which shows what posts were assigned to each category. After reading through the list and figuring it out, I was able to fix the text by typing queries along the lines of:
UPDATE wp_terms SET name='Motive Force', slug='motive-force' WHERE term_id=8;

Since I brushed over it above:
To find a cached version of your site on google, just search for “site:YOURDOMAINNAMEHERE.com”, then click on “Cached” next to any of the results. They clear those out after a while, so don’t put it off!

Today I upgraded the blog to the newest version of WordPress because it was so old it couldn’t even run Akismet (the anti-spam plugin). In the process, it deleted the names of all of my categories. If you see numbers where word should be or blank spaces where categories should be - this is hopefully just temporary.

Sorry!

When you’re running a long command on a terminal over SSH, you may end up wasting a great deal of time checking back repeatedly to see if the process is complete. A quick alternative would be to make the shell beep when it’s complete. Assuming you are running a script called “longScript.sh”, then simply typing a line like:
> longScript.sh; printf \\a
Will cause most SSH clients to beep after longScript.sh is finished running.

Hope that helps!

I recently discovered a very shady practice in AdSense for Search where they make you believe you are earning revenue, then quietly at the end of the month they deduct a fee at payment-time equal to the amount you earned (ie: you net nothing).

It’s pretty hard to believe, to the point that you may think I’m confused. However, I’ve checked. I’ve double checked. I’ve uttered expletives, calmed myself down and checked a few more times. Here’s the money shot:
I checked back and they’ve been doing this for months. I checked the documentation that’s linked to from that “Fees” line, and long-story-short: it’s a hidden fee affecting “less than 1%” of users which they don’t warn you about nor give an explanation of how to avoid it. It’s also barely mentioned at all in their Terms & Conditions (section 11, second sentence) which, of course, protects them legally but is an extremely shady business practice even if they had explained it in detail in the T&C.

This post probably isn’t very eloquent at the moment. It’s hard to write intelligently with steam pouring out of your ears. So without further ado, here is a copy of the letter I sent to their Customer Support which has much more information.

Open Letter to Google AdSense

I’d like to express my extreme dissatisfaction with the hidden AdSense for Search fees. I feel that the way in which you handle these fees is disingenuous on several levels and jeopardizes your integrity from a publisher’s perspective.

There are several areas where I think you guys really dropped the ball:
- When we sign up for AdSense for search there is no visible indication that there was even a possibility of getting charged. Burying it deep in the Terms and Conditions isn’t sufficient when it is completely contrary to the ad-network model of publishers-use-your-network… publishers-get-paid.
- There was no warning at all while this was happening. I very regularly check my stats on AdSense for the day and the month and there wasn’t any indication that each day my revenue was in fact zero, not the number I was being shown. Even if users are aware of this policy, they won’t know that they’re not making money until after the month is over and you decide to take it away.
- There is no easy way to notice this discrepancy after the fact. Old monthly reports will STILL show fake revenue (”fake” because you may take away every cent of it), and they do not show the deduction. Even upon looking at the payment history, there is no indication on THAT page of the deduction unless you drill down to yet ANOTHER page.
- There was never a notification that this was happening even though your FAQ alludes to this being extremely rare (”less than 1% will be affected” in answer 9890), my notification settings are set to the highest level possible, and this program even had the nerve to send me an ad on “New and improved AdSense for search” so I could generate even more money for your program of which I would not see a cent.
- The documentation mentions that some magical equation may take our money at the end of the month but does not provide any transparency into 1) what the metrics were that caused this or 2) how to avoid having this happen again. For what it’s worth: I’ve looked back through my payment history and every cent of revenue from when I first started AdSense for Search back in September has been taken every month, so this isn’t a fluke. Something about my site is angering your algorithm.

I feel robbed. Worse than just tricked; I feel like you stole money (hundreds of dollars) that I had every reason to expect was earned and was going to be direct-deposited.

One of AdSense’s differentiators is that it’s simple and turn-key so that publishers can focus on creating a valuable website instead of dealing with ad sales or applying to ad networks. If publishers have to spend time pouring over every available report to watch our backs for hidden fees, then your offering would clearly fall short in providing that value.

Based on this lack of disclosure, I think your fees should be re-evaluated. I ask that you let me know what you think is fair.

*Shuts eyes & takes a deep breath*. Hokay, I’m back now. I’ll keep you updated when/if I hear back from them.

It’s been a pet-peeve of mine for some time now that creators of web-apps (like myself) haven’t been holding ourselves to the same standards as creators of desktop apps. One of the major justifications for calling our products “applications” (as opposed to just “really good web sites”) is that we’re claiming you get the same level of functionality as a comparable downloadable desktop version.

Currently this just isn’t the case. When is the last time you went through the following process to update a program on your desktop:

1. Check the creator’s website frequently to see if there are updates
2. If there is an update, go through a directory of a bunch of different versions and figure out which one you want since “most-recent” and “stable” are almost always different versions
3. Download the files
4. Unzip the files into a separate folder
5. Make backups of your existing installation if desired
6. Copy the files from the new folder into your existing installation

Seriously? This may look absurd to most users (I hope it does), but this is exactly the status quo for updating web-apps. I don’t know of a single web-app with acceptable update abilities. The closest thing is the DreamHost one-click-installs/updates of applications, but that’s something they have to write themselves for each app and isn’t part of the applications themselves.

I recently found out that I’m not alone in desiring this: it is currently the most-requested idea on WordPress Ideas site (which is like Motive Suggest for WordPress).

It’d certainly be complex to write an auto-updater since there are so many different systems the apps can be installed on. However, desktop installers were never easy to write either and we’ve come to expect them. My suggestion is that web-apps have one setting that allows complete auto-updates (but this option is disabled by default). The more normal use-case would be that the app checks a webservice to figure out if it’s up to date. If not, a very visible “upgrade now” button would be shown to the administrators when they log in. This button could be highlighted yellow for normal updates and red for security-critical updates. For extreme security-crucial updates, the app could actually email its administrators to notify them that there is a critical update and should have a link to kick off the upgrade. Since each version of the app itself is sending this alert, the administrators don’t have to join anyone’s mailing list to get these alerts (so they maintain their privacy).

Just a thought. I’m anxious to release a downloadable framework now just so I could include updating and hopefully start a trend (OffhandWay, MotiveSuggest, SiloSync?). Also, if you see instances of this type of feature in the wild, please let me know!

The overarching theme of the SiloSync Survey was that people really want to back up their data from Facebook!

Since the results were in order of preference and the questions could all have multiple answers, I made a simple scoring scheme. I assigned 5 points for the first answer, 4 for the second, etc.. The 5th answer and beyond were all 1 point.

The results:

What would you use SiloSync for?

1. 24 points - Backing up data
2. 13 points - Leaving a social network if it gets shady
3. 8 points - Keeping profiles on multiple social networks in sync
4. 4 points - Quickly joining new social networks w/o re-setting up

Those results were amusing since the whole mainstream-blogosphere data-portability argument seems to be focused on the lowest-ranking result. Basically small startups want to take away Facebook’s edge so they’re pushing that one the hardest (I don’t blame them). It’s good to have some actual input from users so I can make SiloSync the best possible!

Onto which social networks the respondents were interested in:

Which sites would you like to be able to grab data FROM?

1. 27 points - Facebook
2. 11 points - Wordpress
3. 7 points - LinkedIn
4. 6 points - Blogger
5. 5 points - GMail & Flickr
6. 3 points - LiveJournal, Voicemail
7. 1 point - Twitter

No suprise that Facebook dominated that one. MySpace got served: no one even mentioned it. MySpace also got ignored in the next section on exporting data. However, keep in mind that this section isn’t too important since the people surveyed indicated that they are much more interested in getting their data out than sending it other places.

Where would you like to be able to export your data TO?

1. 17 points - LinkedIn
2. 11 points - Facebook
3. 8 points - Twitter & SiloXML (ie: an open XML format)
4. 5 points - Gmail
5. 4 points - Pedlr
6. 3 points - WordPress
7. 2 points - Blogger
8. 1 point - LiveJournal

There you have it! Everyone hearts Facebook a whole-bunch but doesn’t trust them one iota with their personal data. It’s a good thing too because they’re riddled with security flaws (but that’s for another post) and pull stuff like the NewsFeed & Beacon.

Big thanks to everyone who took the survey for me. It helps a ton when approaching a project this large to know which parts are desired the most.

… guess I’d better get to work on SiloSync now, huh?

For the fourth DevHouse Pittsburgh event we’re doing a BBQ from 2pm to 8pm today. See the site for more details: http://devhousepgh.org!

…also, if you haven’t yet: please take a second to fill out the 3-question SiloSync survey. Thank you!

SiloSync is a sizable undertaking and there are a number of different potential places to start from. I want to make sure I have a decent idea of where the demand is, so I’ve put together a quick 3-question survey. Please take a minute to fill it out for me! (you can be anonymous if you’d like)

To answer, please just leave a comment. I’ll leave my own answers in a comment as an example.

Question 1: What would you be most anxious to use SiloSync for?
A. Syncing up data & friendships between profiles on different sites so that they are all up-to-date.
B. Backup up data (photos, etc.) & friendship connections so that they never get lost and are all in one place.
C. Changing services if one of them does something unacceptable (along the lines of the Facebook Beacon debacle).
D. Quickly joining new services w/o the trouble of re-finding everyone and re-typing everything.

For this, please just type all of the letters you are interested in from highest-to-lowest

Question 2: What services would you most like to be able to pull data into SiloSync from?
(examples: do you want to pull your data from Facebook, Flickr, LiveJournal, Wordpress, Twitter, MySpace?)
Again, please type the most-desired first.

Question 3: What services are most important to export data to?
(examples: do you want to send-data-to/sync-data-with Facebook, LinkedIn, Twitter, or a bunch of new and exciting social networks we don’t know about yet?)

Hopefully that was quick! Thanks for taking some time to help me out :)

There was a bit of a mess over on LyricWiki off and on for a few days. The culprit was a known bug in mySQL which messes up master-slave replication if you run out of hard-disk space (which you will if you’re using master-slave replication).

The preventative solution is to set up a daily cron-job which will find out what log the slave is using and delete all of the binary log-files that are older than that file. The alternative is an immense pile of unneeded files which will eventually cause you to run out of space and completely break your replication. To give you some idea, we filled up 100gigs of log-files from LyricWiki (which has hundreds of times as many reads as writes) in about 2.5 months.

Hope that helps!

UPDATE: I just wrote this script and figured I’d release it publicly to save others some time. You can get the code from deleteOldBinLogs.txt (that’s just a .txt so you can view the code… save it as a “.pl” file). Once you’ve filled out the “configuration” part at the top and have uploaded the file to the “/root” directory on your Master database server, add this line to your crontab file (by typing “crontab -e” into the command line):
0 4 * * * perl /root/deleteOldBinLogs.pl
That will make the script run at 4am each morning.

The talk I gave this week on SiloSync at Pitt was a fun venue. Their Lunch-and-Learn series is a really cool idea and sounds like it’s getting even more interesting. Next month’s talk is going to be done by a VP from Sun Microsystems. Prior to presenting, I jumped back into the SiloSync code and wrote the beginnings of the importer for Facebook.

As a side-note: one of the things that’s fascinating about this project is that I get to see all of the half-implemented security that different sites use. LiveJournal had a secure way of sending passwords, but shockingly stores passwords as plain-text (a big security faux-pas). Similarly, I saw some left-over fields in Facebook’s login form, but it appears that they just punted and used https (a secure web connection using SSL encryption) to just encrypt the whole login.

Back to the crux of this post: I’ve been rather tempted lately to actually finish SiloSync - which I had previously shelved in hopes that Open Social and other big-name initiatives would fix the problem (they didn’t). Google, Facebook, and MySpace have all announced fake data portability initiatives in the last week or so, which shows that if we want our data to be free, we’re going to have to take it (see my previous post on freeing the social graph for why this is important).

I decided it would be best to make a habit of posting my slide-decks when I present (I appreciate it when other people do that), here are the PowerPoint and Open Document (Open Office) versions. In the process of making the presentation, I ended up creating a visual representation of SiloSync which I think does a great job of summing up the whole idea for someone who hasn’t been exposed to it yet. That’s the picture above and to the right… click it to see the full-size version.

Interestingly, with these effectively useless announcements from the major Social Networks, a lot of non-technical press has been declaring that data is now free. Okay, cool, let’s all go home.

Fortunately, most of the technical press is calling them on it. Everyone from TechCrunch to David Recordon (of OpenId fame) is telling it like it is.

If you are interested in seeing SiloSync pushed to fruition (more than you’re interested in seeing Motive Suggest or doItLater v2.0), let me know so that I can weigh off the interest between the several projects competing for my time. Also, feel free to leave comments about your thoughts on the various “fake” data portability. This seems to be the topic which always gets the most vocal response on my blog.