UPDATE Apr 8th: the server has been renewed for one month! thanks for your suppport!

In these days is trembling the future of giss.tv as a server.

Giss.tv is many things, not only the service that gives many the possibility of streaming their radio to the whole world, using the services of the icecasts that want to join the ring.

Is also the software development that allows to maintain the web interface, the icecast branch that starts forwarding to the slave servers the clients after its minimum its reached… and is all this other servers that are waiting for extra streams to serve.

During these years the giss main server was payed by el hangar, but this year they are not sure of doing it.. and the las payment expires on april 10th… that is three days!

If you want to know more about this project, you can read this article

If you feel this project should continue, please mail el hangar with your opinion: info(at)hangar.org

If you want to collaborate with the project, you can join the list stream-ring(at)poliforma.org

These days I’ve been playing with Xapian, to add a better search to the Public Domain Works website. We needed a nice search box to be able to find something on this huge dataset with not a lot of information per record (hopefully this is going to change soon, with the rdf store!). Right now, the site hosts a total of 125318 persons, 299140 works and 12840 items.

For this we are using, as in the open shakespeare website, xapian: an open source search engine that can help you to index any data.

We are indexing a database here, but you can also index text files or i guess even id3 tags.

With the python bindings you can create a xapian document almost from everything. You should be careful just to take a field to use as an index key and that’s it. As I’m working with a database, my index key is the object id on the orm mapper database, but if I was to index files, I could just use the file path as an index key.

So,  in this huge database I have scarce info per record, sometimes just the title for a book, hopefully more stuff like date of publication, description, authors. Many dupes too…

My main search example these days has been “alice lewis carroll“. I wanted “Alice’s Adventures in Wonderland” and “Alice’s Adventures Underground” to appear first on the results.

Why Alice? Because Lewis Carroll was actually the Reverend Charles Dodgson, and the book titles have stemming problems: alice is not Alice’s or Alice.

After a lot of fiddling and reindexing, I decided that I wil index each book with the names and a.k.a.’s of its authors, and all the authors with the titles of their names too. I also added some fields like the url and some extra data to show, so I don’t need to hit the postgres database when doing searches.

For understanding xapian I found this post from Nadav Samet really useful: Pumping Up Your Applications with Xapian Full-Text Search · Nadav Samet’s Blog.

So, to not repeat the work of Nadav, I just add a part of the search class here: the indexer. For each work I have in the pylons database,  I did:

def index_work(self,work=None): """Gets a work object (model.Work()) and returns a Xapian document representing it and a unique article identifier."""   doc = xapian.Document() indexer = xapian.TermGenerator() indexer.set_stemmer(self.stemmer)   indexer.set_document(doc) extras = '' #to create the string we retrieve in the search data = [] #to create the string we retrieve in the search   persons = [] #to create the extra info items = [] if work.title: data.append(work.title) # we index the work title with a double weight indexer.index_text(work.title,2)   else: # many works without name! print "work %s has no title" % work.id if work.original: work.title = work.original elif work.src_id: work.title = workwork.src_id else: work.title = 'UNKNOWN' if work.type: indexer.index_text(work.type,1,'TY')   if work.persons: for person in work.persons: if person.name: persons.append(person.name) #many persons without name! indexer.index_text(person.name,1)   for term in person.name.split(' '): indexer.index_text(term) doc.add_term("NA"+term.encode('utf8')) if person.aka: if type(person.aka) != unicode: for name in person.aka: indexer.index_text(name.lower(),1,'NA') else: doc.add_term("NA"+person.aka) indexer.index_text(person.aka)   # this is what appears in the search results: doc.set_data(''.join(data)) # this are the tags we can search as title:, pd: # see self.search(): indexer.index_text('work',2,'TY') doc.add_term("ST"+str(work.pd)) # this one is the pylons id, to retrieve the # real object whenever we please: work_id_term = 'ID'+str(work.id) doc.add_term(work_id_term) doc.add_value(ITEM_ID, str(work.id)) doc.add_value(ITEM_MODEL, u'work') # any easier way of getting a list of # works/items/persons together than to store the url as a key? doc.add_value(ITEM_URL, str(work.url)) # some extra info to enrich the engine if persons: extras = ' by '+', '.join(persons) if work.date: extras += ' - '+str(work.date) if extras: doc.add_value(ITEM_EXTRAS, extras.encode('utf8'))   # now we update the doc using the id as key. # if the work is not there, it will be created   self.xapiandb.replace_document(work_id_term, doc)   return doc

Next step: setting up an rdf store and indexing it too!

Really interesting poet Amiri Baraka, met him not long ago.
These days I’ve been listening the classes he did at the Jack Kerouac School of Disembodied Poetics. This school started by Allen ginsberg and Anne Waldman is uploading all their contents to the Archive: check out the Naropa Poetics Audio Archives.

Here one of his poems:

Ka’Ba

A closed window looks down
on a dirty courtyard, and Black people
call across or scream across or walk across
defying physics in the stream of their will.

Our world is full of sound
Our world is more lovely than anyone’s
tho we suffer, and kill each other
and sometimes fail to walk the air.

We are beautiful people
With African imaginations
full of masks and dances and swelling chants
with African eyes, and noses, and arms
tho we sprawl in gray chains in a place
full of winters, when what we want is sun.

We have been captured,
and we labor to make our getaway, into
the ancient image; into a new

Correspondence with ourselves
and our Black family. We need magic
now we need the spells, to raise up
return, destroy,and create. What will be

the sacred word?

I leave you here some excerps of the class, for you to leave on your playlist. This and more you can listen at cryptodrunks radio.

amiribarakaclass-excerpt

amiribarakaclass2

amiribarakaclass3

amiribarakaclass4

amiribarakaclass5

More info: Amiri Baraka on Wikipedia