At last. Caching has been carefully designed, developed and deployed and is properly running under a cron every 2 hours. I realized that while checking for already cached entries, it is not necessary to compare the data for altered ones as it doesn’t have any purpose (yet) on how we display the cached data. We do not yet have purposeful features that will alert of modified entries during the past week. So only new ones are added.

The ‘Latest Updated’ feature of the site is now running smoothly, except for some entry body text character encoding problems. I will be working on it every now and then to fix it and other tweakings, such as truncation should end with the end of a word not between one. The displayed times are in RFC 2822 formatted date in GMT.

Just finished coding the final cache update script. Once run, it will go through each blog in the database taking the feed URL, read the feed, go through each entry and check the date of each one of it. If the date is less than eight days, it checks if the entry has already been cached. If so compares the any difference and updates the cache. Otherwise it adds the new entry to the cache and continues the next item, and next blog.

Once fully tested and if it fits the situatation, the Latest Updated feature development will begin, which will make use of the data collected by the cache update script.

Now Shahee needs to check if the server provides crons. If it is available, you need to set a cron to run the script every 6 hours. So we can have a really updated cache.

It may be a nice idea for the mvblogosphere dev team in male’ to go for a coffee, and talk about everything else except the project. And may be post a summary of the meeting. Specially about the taste of coffee. How about call each other and set a time now?

Thanks for letting me onboard. My part is to develop the ‘Latest Updated’ feature of the system. First, Simon has to complete the required RSS Parser class which in turn is utilized by the RSS Seeker class.

What I came up with is, the system needs to keep a cache of the updates from blogs in a separate table. For this purpose, a database table and a cron has to be setup to automate the process (cron availability is necessary). It will run to Cache any updates and Dispose any Obsoletes.

The cache has to have an expiry period which myself and Senn thought of as 1 week. The cache is necessary to keep an internal history of updates. A set of records to compare any new results and normalize the data. From there onwards, we would have a set of data that we can manipulate and display in various manners for displaying and sorting updates from the relevant URLs.

I just realised that along with RSS Auto-Discovery the class (RSSseeker) needs functionality to detect and retreive the “title” and description of the blog. Currently there is no way of getting this information and it is also required for the OPML output.
This can be easily done by implementing an RSS parser (RSS/Atom etc) into the class.

Work, therefore, is still on-going and the features mentioned in the earlier post might not be available that soon.

mvBlogs.org is going to have RSS Auto-detect and OPML output capabilities. So hopefully in a couple of hours (given that I’ve yet to sleep) it should be up and running.

RSS Auto-detect will be added to Submit Blogs area and should, as it’s name suggests, automatically detect embedded RSS feeds in URLs.

Next is OPML output and it should, if properly implemented, allow visitors to export the entire list or a select list of blogs as an OPML file. This format is commonly used by RSS / News readers / aggregators (such as Bloglines / Newsgator etc) to exchange lists of RSS feeds.

I’ll be posting updates on the progress of these two tasks and the PHP class file here shortly.

mvblogosphere is an attempt to track, map and analyze the exponentially growing maldivian blogopsphere, in order to draw a larger portrait of the social, cultural, political and economic structures it represent.

The mvblogosphere project got initiated when Senn (visual artist / network technician) compiled a list of the blogs he was able to visit from one random point and showed it to Shahee (media designer). With a collaborative effort, they were able to come up with techniques to visualize the blogosphere.

The visualizations are generated using a ‘dot’ generator programmed by Senn. Different ‘dot’ renderers and photoshop is used for the final output. Ideas about programming a ‘crawler’ to gather more data is being developed by Simon (programmer).