Strip HTML tags from a string, Python edition

Obtain Beautiful Soup.

from BeautifulSoup import BeautifulSoup

''.join(BeautifulSoup(page).findAll(text=True))

where 'page' is your string of text and HTML.

I'm not a pythonista, there might be a nicer way of doing it (Beautiful Soup is a lot of overhead). Might want to expand on this a bit to make sure spacing is handled OK, you can keep certain tags etc. etc. Feel free to post corrections or better suggestions in the comments.

Just don't use one line <(?:.*?)> regular expressions. No, really.

Academia.edu

I like academia.edu. The academic family tree idea is pretty cool (I know that the concept has been around for a while in various guises but their implementation is pretty slick) and I like the fact that new visitors can arrive and be interacting with the site within minutes. It's also nice to see an academic networking site that, well, doesn't look like Facebook.

I'm also impressed by the speed at which they've been throwing up refinements and bug fixes... and by the adverts on PhD. Canny marketing (good work poorly paid but well fed intern)! The academia.edu team are a smart bunch of people which is probably how they got funding in the first place.

For balance what's not good about it? The flash freezes my mac on an empty cache... and the .edu TLD is really only for educational institutions, not commercial enterprise (vetting only started in 2001, academia.edu was first registered back in '99). Tsk! Ironically my other bugbear is that I can't join properly because I work for a commercial enterprise and not an accredited educational institution.

Disappointed with Popfly

Popfly is the mashup editor that Microsoft released last year. The idea is good. The 3D graphics are good. Silverlight is a bit buggy in Firefox (sidebars don't always redraw properly) but that's OK.

If you're going to create a web 2.0 mashups builder, though, don't you think it's be a good idea to provide some Atom support?

Meta-analysis

The journal platform team here at NPG just rolled out machine readable metadata for the papers we publish in Dublin Core, PRISM (good PRISM, not to be confused with evil PRISM) and Google metadata formats.

No more scraping to automatically get the citation for a paper, it's all in the HEAD:

<meta name="citation_journal_title" content="Nature" />
<meta name="citation_publisher" content="Nature Publishing Group" />
<meta name="citation_authors" content="Paul Schenk, Isamu Matsuyama, Francis Nimmo" />
<meta name="citation_title" content="True polar wander on Europa from global-scale small-circle depressions" />
<meta name="citation_volume" content="453" />
<meta name="citation_issue" content="7193" />
<meta name="citation_firstpage" content="368" />
<meta name="citation_doi" content="doi:10.1038/nature06911" />

Useful for apps like Zotero and Connotea (which before now downloaded two files each time you bookmarked a Nature paper: the page itself and then the linked EndNote file to parse).

The metadata will be there for all papers going forward and back through some of the archives.

For fulltext indexing of papers behind the paywall you can use the linekd OTMI file (I only just saw Twease, which does just that) although there's only OTMI for Nature papers at the moment, I think.

Lastly at some point in the future we're aiming to put XMP metadata in our PDFs, which should make it much easier for scripts and applications (like Papers) to look at PDF files on your filesystem and work out what they represent.

Nice work Pedro!

Noticed while leafing through today's Nature that Pedro has a paper out (Isalan et al., Evolvability and hierarchy in rewired bacterial gene networks).

There's more on this over at Public Rambling.

Ian owes me a pint

(update: Gavin Bell at Nature gave up one of his app spots so that I could put this live, which I did: only to discover that Google App Engine is even more unforgiving of timeouts than Facebook. Currently trying to work out how to make the bookmarking process, for now it doesn't work very well. Also the search is broken, though that's Google's fault and not mine.)

I bet Ian earlier that I could rewrite Connotea on App Engine in six hours. I can't remember why. Probably ego (mine, I mean). He didn't actually bet me a pint but he should have done...



... because the original estimate was a tad optimistic (ahem). After twelve hours I've produced pycite, though, which is pretty good going I think. I'll admit it: Python is actually very cool.

pycite is three hundred lines of logic and a set of html templates that implements a (very simple) social bookmarking service. Sadly I don't actually have an App Engine account so it's not live on the web anywhere (I'll buy whoever does have an account and puts it up first a pint - let's spread the love), you'll have to download it and run it locally to see it in action.

What you can do with it:

• run it without owning a server of your own


• log in with your Google account


• add new bookmarks (the citation will be collected automagically)


• view everybody's bookmarks


• filter bookmarks by user:

  http://path.to.pycite/users/bob.smith



• and by tag:

  http://path.to.pycite/tags/diabetes



• and by user and tag:

  http://path.to.pycite/users/bob.smith/tags/diabetes



• and by keyword (the full text of each bookmarked page is searchable):

  http://path.to.pycite/users/bob.smith?q=t2d



• get atom feeds for all of the above

What you can't do with it (yet):

• edit or delete bookmarks


• anything else

I've put it all up on Google Code. It's fairly straightforward stuff so if you've got any brilliant social bookmarking ideas then go for it. Send me an email and I'll give you write access to the subversion repository.