Feed on Feeds 0.1.6

0.1.5 still had some pretty major bugs. I’ve fixed those, and tested this relase pretty thoroughly. Hopefully this one has no “SEVERITY ONE” issues.

By the way, I’m very very impressed with the “many eyes” effect on these last few releases. The community of FoF users found all the bugs almost immediately… within hours! And in the middle of the night! (at least, it was night here). And, had not only had they posted good bug reports to SF, but they even found the root causes, and produced patches! I wish the QA group at work was one tenth this good.

“Dear Steve, noticed you just checked in Boring.java. You’re using an unsynchronized Hashmap at line eight billion and seven. Funny thing, that will work most of the time, but it turns out that in cluster mode it will cause silent data loss. Checked in the fix for you, k thx bye.”

So thanks, everyone, for bearing with us during the technical difficulties, and helping out to solve them!

Feed on Feeds 0.1.4 is here!

But you don’t want it. You want Feed on Feeds 0.1.5.

Here’s what I ended up with for features:

• CSS / XHTML. The only remaining table is the feed list. As I said before, don’t worry, the much praised FoF look and feel has been scrupulously replicated in CSS. Or, you could use…
• … The new frames-based, one page viewer.
• Better charset handling - I’ve hacked MagpieRSS so that it does its very best to always return UTF-8, and that is used as the internal format, as well as the charset for all output pages. Kellan is looking at these changes now to see if he wants to add them, or something like them, to the mainline Magpie.
• By default, read items will be purged completely (during the update) from the DB after 30 days. You can adjust this timeout, or shut this off entirely to go back to the old “never delete anything” behavior. Warning: If you’ve been using FoF to build up some kind of huge awesome database of feed items that you are very proud of, be very careful with this new version! You could very easily find them all deleted.
• The cache directory is located in a smarter way. People trying to include FoF or call it from the command line should have better luck
• Continuing code cleanups: more and more “logic” pulled into init.php, leaving the viewer pages purely “presentation”. More careful about namespace pollution, everything is prefixed with fof_ or FOF_. FoF should be much more “includable” now.
• The meagre beginnings of a PHP API: look at fof_get_feeds and fof_get_items.
• I think this time I didn’t completely screw up checking my changes into CVS. I did last time, nobody noticed.

To upgrade: install to a new, clean directory, and copy your settings over from your old config.php. There’s a lot more stuff in there now, so don’t just copy the whole file. And again, if you don’t want all your old, read items to be deleted, be very careful. Make a DB backup or something.

Sorry to keep you all waiting, but 0.1.4 is almost done. It’s taking a bit longer because for some reason it has twice as many features as I expected. They just keep creeping in there somehow: CSS, XHTML, much better charset handing, refactoring, includability, easier to use PHP API, new frames based display (shown above, but don’t worry: the award winning original FoF look and feel has been painstakingly replicated in pure CSS and is still included), database size management, …

I’ve got a pretty busy weekend, so no promises, but maybe on Sunday.

I completely agree with this. On Safari, even UNDO doesn’t work in a textarea! Every post is one miskeyed “select all” away from complete destruction. At least WP lets me save and keep on editing…

(via)

So is fsockopen just completely hosed in PHP 4.3.4 or what? I have a friend who was happily using FoF for a while, then it stopped working. His host is running PHP 4.3.4, and I suspect they just recently upgraded, and I think that broke fsockopen (which Snoopy uses (which Magpie uses (which FoF uses))). I’ve searched and found some bug reports on fsockopen in 4.3.4, but not quite with the same symptoms. Here’s what I’m seeing:

• fsockopen to ‘127.0.0.1′ or ‘localhost’: works!
• fopen to an http URL on another host: works!
• fsockopen to another host, by name or by IP: fails! No error message is printed or logged. In fact, the script dies so completely that the browser gives a “This document contains no data” error. If you telnet in and type the HTTP request yourself, the server closes the connection before sending back any headers or anything.

At first I thought maybe his host had disabled fsockopen or maybe banned connections to other hosts. But fsockopen is not disabled, it works when connecting to localhost, and fopen works when connecting to any host.

Is this this bug? The symptoms aren’t quite the same. Is there any hope? Any workaround? Or should I just advise him to ask his host to upgrade to 4.3.5?

Come and get it!

New: Various bugfixes, MagpieRSS 0.61, one click mark all as read, Safari speedup, and for you feedaholics: the much-requested ability to flag all the items above the current one. If you’re faced with hundreds of items, and start reading them from the top down but then need to do something else, you can click one link to flag the current item and all the items above it, and then mark them all as read. You can come back and read the rest later!

Safest upgrade: extract to new, empty directory and copy your config.php over. Also should work: extract right on top of old version, then restore settings in config.php. No database tables have been changed.

This is not the fabled multi-user, category enabled, refactored, CSS/XHTML nirvana release. That one is still a ways off. I think a better way to get there will be a set of smaller, more limited releases, rather than trying to do it all at once. So here you go, and more coming soon!

I’ve been hacking on WordPress a bit to customize it for my needs, and I thought I’d share a few things I’ve “discovered”. These may be obvious to you, but they took me a while to figure out.

Logging

I’ve never figured out a good way to do logging from PHP. Sometimes you can get some insight into what’s going on by printing out some comments to the outgoing HTML that a view source will then reveal. But that doesn’t work in all cases: What if there’s something funny going on in the processing of a new post? There’s no place to print to! Actually, there is. Use STDERR!

$stderr = fopen('php://stderr', 'w');
fwrite($stderr,"Logging statement: value is $value\n");
fclose($stderr);

On my host, this works. There is a directory where the output from STDERR is captured. I can post a story, then look in the logfile and see what happened. It helped me debug this quite easily.

wpdb SAVEQUERIES

WordPress has the built in ability to log all the SQL queries executed while building your page, which I happened to notice while looking at the ‘wpdb’ class. Here’s how you can use it:

Step 1: Add define('SAVEQUERIES', true); to your wp-config.php. This will cause all queries run by the wpdb to be appended to an array that you can get at later.
Step 2: Print out the queries somehow. I’m printing them out in an HTML comment at the end of my main template, like this:

foreach ($wpdb->savedqueries as $q)
{
$q = trim(ereg_replace("[[:space:]]+", " ", $q));
echo "$q\n\n";
}

Now I (and you) can see exactly what queries are being executed, which may help answer some questions about what’s going on in the guts of WP.

Today’s word of the day is Geek, because I have had this story of geekism on my mind. Where I work, we are geeks, mostly. One of the geeky things we do is come up with “code names” for each project we work on. There’s actually a good reason for this: marketing will change the actual name of the product 10 times while you develop and test it, and then after you pass the point where the name can not possibly be changed, 10 more times. Then in the next version they’ll rename it more. To escape this cycle of pain, we use code names, which need never change. The product name is set at manufacturing time by some configuration file, but all the internal documentation and source code refers to the code name.

The second level of geekism here is that we name all our projects after colors. (everybody needs a theme, for instance Intel uses rivers and towns in the Pacific Northwest) Previous projects have been Gunmetal, Daffodil and Magenta. Current projects include Quicksilver and Cyan. Coming up soon will be Canary. Some purists out there are now thinking “WELL SOME OF THOSE ARE HARDLY COLORS.” …You need to learn to think outside the box.

The THIRD level of geekism is that we create silly logos for each codename that we use on internal documentation and sometimes tchotchkes. For the project I’m working on, Cyan, the details of which are unimportant, but I can assure you are incredibly boring, I created the perfect logo. I sent it out in an email. I got lots of responses. They all hated it! Not enough “Zing”, one email actually dared to say. People began to do Google image searches and send back alternatives nicked from the web “How about this one?” “OOooooo I like that one!” “Hey, this one has a funky font!” “Look at this funny album cover I found.” And so on, and so on.

We ended up using one of those other logos, leaving mine consigned to the dustbin of history (along with 22,700 other things, according to Google). But because of pride in my fine logo-geneering skills, stubbornness, or maybe plain stupidity, I decided: This will not stand. I’m not going to let my fine logo go to waste just because the people I work with (to a man: even the art designer guy) can’t appreciate it. So I’m working the other end of the lazy-web: Post the lazy-web response now, and when the lazy-web request comes in, the answer will be waiting here.

So here it is. Need a logo for your project/product/company/clothing line/floor wax/dessert topping? Is it called Cyan? USE THIS ONE! IT’S FREE! (note: logo makes more sense if your previous project/product/company/clothing line/floor wax/dessert topping was called Magenta)

Disappointed? Now you know how the people I work with feel. Every day.

First post: pretty good, human voice, posted by an actual person

Second post: not so much (although the possibility of working for Google in Japan is, as they say, wicked cool)

June 28th 2004: Steve Jobs introduces Mac OS X 10.4, known as “Tiger”.

ALSO ON JUNE 28th: Sun introduces Java 1.5, also known as “Tiger”.

This is sort of old news, but I finally read this presentation today and there’s a reason why it’s linked up everywhere. How do you set up a weblog hosting site that serves 2.4 million users (who read AND write), and deal with peak hit rates of 800 per second? Like this: Inside LiveJournal’s Backend (pdf).

Remember Data’s “scanning for lifeforms” song? Here’s the remix!

Anybody else notice this yet? Is it new? Numrange Searches: Numrange can be used to specify that results contain numbers in a range you set. You can conduct a numrange search by specifying two numbers, separated by two periods, with no spaces. Be sure to specify a unit of measure or some other indicator of what the number range represents.

For example, shaw brothers 1975..1980

Here’s my other desktop. I made it all by myself.

Same joke as before, comes in any size, as long as it’s 1280×1024.

Using images stolen from ＯＳたん保管庫 (don’t know what that means, but the site can only be described as “operating systems fan art”), I’ve created this desktop, perfect for use on a Mac:

It comes in whatever resolution you want, as long as it’s 1024×768.

LAW OF ADMINISTRATION FOR THE STATE OF IRAQ - FOR THE TRANSITIONAL PERIOD

The people of Iraq, striving to reclaim their freedom, which was usurped by the previous tyrannical regime, rejecting violence and coercion in all their forms, and particularly when used as instruments of governance, have determined that they shall hereafter remain a free people governed under the rule of law.

These people, affirming today their respect for international law, especially having been amongst the founders of the United Nations, working to reclaim their legitimate place among nations, have endeavored at the same time to preserve the unity of their homeland in a spirit of fraternity and solidarity in order to draw the features of the future new Iraq, and to establish the mechanisms aiming, amongst other aims, to erase the effects of racist and sectarian policies and practices.

This Law is now established to govern the affairs of Iraq during the transitional period until a duly elected government, operating under a permanent and legitimate constitution achieving full democracy, shall come into being.

(via The Command Post)

Amazing, in the same day I finally make contact with the guy from the famous Mario Piano video (the comment from “the source” really is him!), I find this video. It’s by KeiicHi, and from the look of his message board, the video has already made the rounds on at least the Asian parts of the internet. It’s a great version of the Mario music, complete with sound effects!

A post from one of his fans on the message board sums it up nicely:

You.

Very Good.

Your hair.

Beautiful.

-Guitarist from the hell in Korea.

And even though I know it’s not going to stem the tidal wave of comments, I’m going to try anyway by pointing to:

Guitar Tablature Here!!!

Ha, the Internet is great. That last post has sent me on an amazing Google-powered journey through Trek Trivia. Everything from “NO KILL I” to “Botany Bay?!!? OH NO!!” to “Queen to Queen’s level three”. I’ve determined what the ultimate Trek trivia question is. Anybody know this one, no cheating?

Of course everybody knows the self destruct sequence of the Enterprise, code one, one-A, two-B, zero zero zero… DESTRUCT… zero, all that stuff. Here’s the question: HOW DO YOU ABORT THE SELF DESTRUCT SEQUENCE?!?!? Do you know the code? What? You don’t? 8… 7…. 6…. 5… GET OUT! GET OUT OF THERE! GET OUUUUUUTTTTT!!!!!

Pop quiz! The Ships Of Star Trek. I got 7 out of 10 on this, even though it seemed like most of the questions were on Deep Space 9 or Voyager, which I never watched.

(I do have to take issue with one of the answers. The Federation class “Dreadnought” is CLEARLY documented in the original version of the Star Fleet Technical Manual, which being published in 1975, I think must be considered canonical.)

Sitepoint on Feed on Feeds: "Simple, but in a good way."

The computers that power Spirit and Opportunity are BAE Systems RAD6000’s. They run at 25 Mhz (not a typo), and cost $300,000 (also not a typo). Here’s two articles about them.

I guess what you pay for is the radiation hardening, the greater temperature range, and the higher mean time between failures? The RAD6000’s MTBF is rated at over 375,000 hours. I can find some server-grade PC motherboards rated at about 30,000 hours. I don’t even want to know what consumer stuff is rated at.

Want to read a treatise on the 18 different types of spaces in Unicode? How about on the 16 different types of hyphens and dashes (and their line-breaking properties)? OK, maybe you DON’T want to read about those topics, but need to? Try Jukka “Yucca” Korpela’s Characters and Encodings.

There’s lots more good stuff at his site, including this page of millions of useful links and tables related to web authoring.

(via dive into mark)

Tiger is getting closer. Here’s the docs and here’s the code for Beta 1.

FoF + MT = reBlog

What is a reBlog?

A web site republishing the best blog posts from around the web. Eyebeam’s R&D department created this site as a proof-of-concept for reBlogging.

Who is reBlogging?

We will periodically select a guest reBlogger to pick their favorite posts and links on art, technology, and culture.

How Does it Work?

We developed a system that aggregates RSS feeds from around the web, a simple interface for selecting our favorite posts, and a hack that uses Movable Type to republish the content. [see the hack]

According to Dave’s new “RSS validator”, my Atom feed is valid RSS. Strange.

Feed on Feeds now supports Atom! I was able to do it in about 3 lines of code, using this experimental new version of MagpieRSS.

To upgrade: Download FoF 0.1.2, unpack to a new directory, and then copy over your config.php.

Is this the dorkiest thing ever?

No. This is.

Looks like the Minitrue got to the famous Apple “1984″ ad.

superbowl 1.6.84 commercial ipod nonreference doubleplusungood rectify

Want Atom support in FoF? Scott Hanson’s got the patch! I haven’t tried it, but it works for him.

JAG mentioned that P=NP, if your processor speed is increasing exponentially. Like the one you’re using right now! Don’t you buy a new computer, with an exponentially faster processor every few years? QED. Cool. I don’t see how P=NP implies that AI is possible, though.

He mentioned that this idea was originally from a guy named Ed Fredkin who writes papers about this kind of thing all the time. Here’s another site about these kind of ideas. This Fredkin guy seems like a “real deal” version of Wolfram: He thinks a lot (and writes actual papers, not just long, vague books) about what it would mean if the Universe is actually, at the lowest levels, discrete and algorithmic.

“Here’s the world famous computer programmer, placing a test order at the dot com site he works for, at Y2K + one second. (note the ball that just dropped on the TV) Will it work? WILL IT WORK!?!?!?!”

Actually, it didn’t work! The order showed as placed on Jan 1, 1999! But it wasn’t a Y2K problem. Some bonehead had used the unbelievably arcane and incorrect “IYYY” date format on the order confirmation page, which uses all these crazy week numbering rules (defined by ISO8610) to determine that it really isn’t the new year if there hasn’t been a Thursday yet. What? Who wants that? It had probably been broken since the site existed and nobody had noticed. So I quickly changed it to the proper “YYYY” and all was well.

Any of the other geeks out there have a good story from that (not very) fateful night?

Knight Commander of the Order of the British Empire!

His only regret: those pesky two slashes.

According to a message that just came in off IP, here’s the latest US computer system that has been hacked into by Chinese spammers:

The USS San Antonio! Hacked before even leaving dry dock.

Remember that new syndication feed format project thingy? It’s still going, and there’s a new release of some specs that Mark has documented nicely. For fun I’ve created an Atom 0.3 feed. b2 and (maybe) Wordpress users can find the template here. It has a good chance of working for you as-is, if you want to play this new game.

This is one of the geekier things to come along in quite some time. Get your personalized future light cone RSS feed!

Good thing this is now available, it turns out my future light cone will intersect with Beta Comae Berenices in just a few weeks.

Package tracking? And patent lookup?

They’re also working on a new clean look, which will be the first portal redesign in Internet history that actually makes the ads LESS prominent.

Click on this button, and then check the address bar. Are you where it says you are?

Where do you want to go today? (Works in IE only)

This is bad because I could have tricked you into thinking you were anywhere I want, and then created a page that looks indistinguishable from the real site. I could have made a fake CNN, or Paypal, or anything. And it didn’t have to be a button either, I could have done it as soon as you came to this site. The next Google search result you click on could do this.

More on this vulnerability.

I have implemented Bayesian spam filtering for this weblog. I have no idea if it is working. Well, I think it’s working. I’ve trained it with all the comments in the database. They’re all considered not spam, since I always delete spam when I see it. Then I had to go back into some archives and find spam comments that have been posted, to teach it what spam tastes like. I could only find 19 such spams, which I have a feeling isn’t quite enough.

When asked to score the good comments in the database, it is currently giving scores like .00000000000000000000001% chance of spam. And when I ask it to score some of the spams I’ve used for training, it says 100%. So it is not obviously completely broken. But I’m not sure what it’ll do when it sees stuff it’s never seen before. I can’t wait to find out… so Allez Spam!

I haven’t thrown the switch that consigns comments that rate highly on the spam-o-meter to oblivion yet, because I’m not too confident in the system. For now, it’s just working behind the scenes, rating and learning.

I first tried to use an existing free implementation of Bayesian filtering in PHP by Loic d’Anterroches. I couldn’t quite get it to work, and it was a little too general, so instead I rolled my own.

I based it on these two articles by Paul Graham, and partially on the Bayesian spam filter for MT. Translating Paul’s oatmeal and fingernail clippings into PHP was…. entertaining:

(let ((prod (apply #'* probs)))(/ prod (+ prod (apply #'* (mapcar #'(lambda (x) (- 1 x)) probs)))))

becomes

for($i=0; $i < $nprobs; $i++)
{
  $ptop *= $probs[$i];
  $pbot *= (1 - $probs[$i]);
}
return $ptop / ($ptop + $pbot);

I’ve got it integrated into b2, but I’m not going to “officially” release the code until I know that it’s working and useful for comment spams. Of course, those of you who just can’t wait, want to see my horrible slapped together code, and know how to use my view-source feature, be my guest. If it does prove to work, I’ll release it as a b2 hack, and hopefully the Wordpress guys will like it. (I really have to upgrade to Wordpress one of these days…)

(By the way: While I was working on this, I did introduce a bug that completely broke comments for the past day or so. If you tried to post during that time, sorry! It’s fixed now.)

Man vs. Machine - “There are fields, endless fields, where Sodaplay walkers are no longer built… they are grown.” (stole the quote from here, helping to spread the meme)

National Nanotechnology Initiative - I had no idea this was a National Initiative. According to Google, there’s about 2000 such National Initiatives that I was completely unaware of.