Remedie

Remedie Users, come chat with me at YAPC::Asia BOF

I'll speak at YAPC::Asia 2009 about Remedie and AnyEvent/Coro stuff. I also plan to host Remedie users BOF so that I can hear how Remedie is used and what you want in the future releases. Let's chat! Add your name to the BOF wiki.

Toggle Display order -- Great for webisodes!

Jesse got a huge TV now and really started using Remedie to watch webisodes like The Guild to find the same annoyance that we've been having for a long time: the default display order is reverse chron and do not play well with the stock episodes, and continuous playback sucks.

So now, it's high time to implement the long-wanted feature:

Go upgrade to 0.6.18 and enjoy toggling the display order, old to new, where appropriate!

Snow Leopard and perl 5.10 compatiblity

Apple's Snow Leopard is shipped and its /usr/bin/perl is now updated to perl 5.10. Remedie Server binary has a hook to detect if it's running under Snow Leopard and use /usr/bin/perl5.8.9 if it's there, so even though the binary is named -leopard.zip it should work fine under Snow Leopard.

You can also build CPAN dependencies and run remedie-server.pl with perl 5.10.0 or later yourself with Mac OS X or other UNIX OS: there's no problem that I've heard yet. The only thing you should be aware is that if you have Remedie install with 5.8 and upgrade to 5.10, you should remove the cache directory to avoid Storable signature mismatch by just typing rm -rf ~/.plagger on Terminal (this might take some time).

So overall, Remedie binary and source code are both compatible to Snow Leopard and perl 5.10.

I myself haven't installed Snow Leopard since I'm a pretty conservative guy on upgrading OS, but if you encounter any issues, please report here or on our #remedie IRC channel. Thanks!

Playback order

I've been receiving a couple of feedbacks and requests of default playback order. Namely, current Remedie displays the feed items as it appears in the feed initially (when you first subscribe to the feed), and then new item will appear on top of the existent items.

This is something you'd expect for most feed readers, but it might be not so useful when you want to do continuous playback on something like Nico Nico Douga mylist, where they have all episodes from the latest to oldest. There's a hack to specify the sort order in the RSS feed for nicovideo.jp and a patch to change the behavior globally (not recommended unless you know what you're doing), but fundamentally this should be fixed on Remedie UI side, so that you can toggle the sort order by clicking tabs. Or even random playback mode to enhance the jukebox-like experience.

I plan to add the ability to sort channel items in-place by mangling URls, like localhost:10010/#channel/5?sort=date&order=asc or something similar in the coming 0.6.x release.

UPDATE: This is now implemented in 0.6.18.

0.6.x updates

It's been quiet since 0.6 was out of door 3 weeks ago, but I've been actually working on some fixes and improvements to Remedie Server most of the time, and made 12 minor updates. Here's the highlight, but you can also go look at the detailed Changes file on git.

iPhone coverflow UI

0.6.4 and later fixes implemented coverflow UI for iPhone. When you access Remedie Server via Mobile Safari on your iPhone, Remedie displays the special iPhone UI with Remote. And if you flip the phone horizontally, the coverflow UI will start. This is just an eye-candy to look through (or possibly show off) your Remedie library but could be a good starting point to use HTML5 and WebKit's awesome CSS animation for possibly more awesome UI. zflow javascript and CSS library is being used to implement the coverflow UI.

Performance Improvements

0.6.0 already implemented event-based programming along with multi-tasking with Coro framework, but in 0.6.5 I optimized more of the HTML and RSS feed parsing code to eliminate slow and memory inefficient HTML::TreeBuilder in favor of LibXML parsers. Also I profiled and removed most of memory leaks happening in the backend code, so it runs much faster and memory efficient.

Periodic Updater

Prior to 0.6.8 you need to manually click 'Refresh' button or type 'shift+r' keyboard shortcut to refresh the channels. Now the server side process periodically kicks the updater every 60 minute, and the nicest thing is, if you keep the Remedie UI open in your browser, it automatically sends the update-start event over Comet channel and auto refresh them with Growl-ish notifications (If your're running Remedie UI on Fluid on OS X it will directly talk to Growl natively). This is very nice to always keep you updated while you're away from your home Mac mini computer.

I have more plans to implement web hooks, superfeedr integration and PubSubHubbub hook etc. to make your feed updates more real-time.

Improved Continuous Playback

typester suggested the current continuous playback behavior, which only works against unwatched (NEW) items, is confusing. I agreed to that and made a big change so a) if you open a channel with unwatched items, you'll only see those unwatched items. But you can click "M items, N unwatched" to show all items. b) Continuous playback will play whatever items you're seeing in the channel view, regardless of whether it's unwatched or not.

This will make it easy to subscribe to your favorite list, like delicious.com/{yourname} or friendfeed.com/{yourname} and continuously playback everything multiple times.

In 0.6.13 I also added a support of pausing and resuming playback with 'o/RET' key. With tools like Remote Buddy or Sofa Control, or with Remedie's iPhone Remote, you can click the Remote's PLAY/PAUSE button to actually pause the playback. This currently works with QuickTime, Flash, Silverlight, YouTube and Nico Nico Douga embedded players.

Known Issue: Bus Errors (fixed in 0.6.9)

Summary: if you encounter "Bus errors" or "Segmentation fault" when adding or refreshing a channel: a) Upgrade to 0.6.9 or later and b) turn off REMEDIE_DEBUG environment variables if set.

We've confirmed that since 0.6 release there are very rare cases that your Remedie Server process crashes with "Bus errors" or "Segmentation fault". There are a couple of ways to reproduce this.

If you set the debug environment variable REMEDIE_DEBUG, Log::Dispatch apparently gets confused with Coro's context switch in the loop and gives a bus error. Turning off the environment variable would be the solution to avoid this.

Alternatively, when you start the server process while Remedie UI is open in your browser, and you add a new channel without first reloading the page would cause the bus error, at least on Mac OS X. 

This has actually been a deal breaker for the OS X first time users since it should always happen and they couldn't add any channels. I tracked down to find some issues with JSON::XS encoded data and perl reference counter with Coro multitasking, so I added a workaround which hopefully fixes the problem. This patch is now included in the 0.6.9 update.

We'll continue to chase this bus errors or segmentation fault bug with Issue #145. If you encounter the problem, reporting it with a reproducible step and/or gdb backtrace would be valuable.

Remedie 0.6 released!

I'm pleased to announce that Remedie 0.6 is now released, and it has lots of goodies. Mac OS X users can download the binary to upgrade, while other users can check out and upgrade its CPAN dependencies with "perl Makefile.PL && make installdeps".

Performance Boost

The biggest update in Remedie 0.6.0 is its much better performance.

Previously if you have 50 channels in your Remedie collection, it might take 3-5 minutes to refresh all channels, depending how big your feeds are and slow your network is. But with 0.6.0 refreshing channels are done with non-blocking IO and multi-tasking programming, so it would just take like 0.5-1 sec per channel, and the best part is that you can control Remedie user interface while channes are being refreshed. The look & feel haven't changed a lot, but this backend performance update is HUGE.

I also made a lot of optimization in the code that does parsing feeds and scraping content to use libxml libraries rather than pure perl implementation, which makes the content parsing much faster with less memory usage.

No (or less) memory leaks

What you wouldn't like with software that runs on your desktop is that when they eat a lot of memory. I can't count how many times I every day need to restart Safari when it hogs the laptop memory and makes the system slow down a lot. Previously Remedie (and its backend Plagger) had a severe bug in leaking memories when you refresh a channel (feed), so every time when you refresh all channels, the memory Remedie server process uses grows up and up. In 0.6.0 most of the memory leaks, at least happening on Perl land is fixed. I still observe a slight memory usage increase when I refresh all channels multiple times, but it's getting much smaller.

Realtime Comet Messaging and iPhone Remote

The benefit of using asynchronous event loop and multitasking thread programming environment is that it not only doesn't block UI, but also allows advanced pub-sub messaging system like Comet. In this release I implemented a very basic Comet messaging channel between the browser UI and Remedie server, and that allows you to use iPhone (or any other Mobile Webkit based browsers like Android, Iris or Palm Pre) to actually "control" Remedie running your browser.

This is a huge UI improvement, and you can now use your iPhone or iPod touch to control Remedie on your TV, while sitting on your couch, without messing with Bluetooth wireless keyboard etc. Daisuke Murase actually made a pretty simple Bonjour browser for iPhone that finds Remedie running on your local network and use Safari.app to launch Remedie Remote, which is very nice.

UPDATE: 0.6.0 binary had a critical bug that it doesn't create the database cleanly if you haven't previously installed Remedie, also it didn't have the fast C based event library, both of which are fixed with the new 0.6.3 binary.

Remedie Internal Talk at YAPC10

I had a pretty good time at YAPC10 in Pittsburgh last week. Great campus at Carnegie Melon University, great people and great talks.

I gave a talk about my experience and technique building Remedie, using HTTP::Engine, SQLite and jQuery. Unlike my talk in OSDC.TW, it was focused on "How" rather than "What", so if you're a perl developer and not that interested in online video sharing but interested in the desktop application in general, this might be very interesting.

It was a 20 minute talk but I had too many things to talk about, so I was running out of time pretty easily but got a lot of positive feedbacks. I'm not sure if audio or video will be available, but I'll post the links if they're up.

Building a desktop app with HTTP::Engine, SQLite and jQuery

View more presentations from miyagawa.

Renaming Channels

Remedie 0.5.9 is shipped with a significant UI update: renaming channels. No more pesky channel prefixes like "Hulu episodes for ..." or "YouTube videos matching ...". OS X users can download the binary.

Binary builds for OS X Leopard users

My perl projects (Plagger and Remedie especially) have been notorious for the difficulty to install because it requires tons of Perl modules from CPAN, especially if you're not a developer. That's because of my strategy to make my applications used by smart people, who can help themselves and knows how and what to fix when things are broken.

But today I decided to make the door open and lower the barrier a little bit: Remedie binary builds for OS X Leopard. Those files are OS X .app bundles, so you can just download, unzip and double click to run the server process (and then you need to access http://localhost:10010/ with your browser). 

The app binary is built using the awesome Platypus tool and I made a simple wrapper to automate the process of bundling dependencies using local::lib and then creating .app bundle with Platypus command line. I hope more perl projects (including my own Github Growler) available as OS X app bundle using this little tool. (I also think about integrating this as plugins for Module::Install or Dist::Zilla)

UPDATE: These binaries (starting 0.5.2 I just released on Tuesday) should now work on Snow Leopard as well.