proxy cache "force reload"

Share your ideas for future releases of nntp//rss.

proxy cache "force reload"

Postby hein on Tue Sep 30, 2003 8:03 am

Hi,

I am route nntp//rss http requests through a caching proxy and there seems to be no obvious way to tell nntp//rss to force a reload. This comes very handy when debugging faulty rss code...

BTW, does nntp//rss respect Expires: HTTP headers?

Thanks for this very nice software!
Hein
hein
 
Posts: 5
Joined: Tue Sep 30, 2003 8:00 am

Postby jason on Wed Oct 01, 2003 11:32 am

The easiest way to force a reload of a particular channel is to bring that channel's configuration up within the Administration interface, and to click on the 'Update' button. This will cause the channel to be repolled on the next iteration of the channel poller (within 15-30 seconds).

nntp//rss follows the Last-Modified and ETag headers for bandwidth conversation. It does not currently respect the Expires header, although I'll certainly take a look into the impact of supporting this within nntp//rss.
jason
Site Admin
 
Posts: 114
Joined: Sat May 03, 2003 10:44 pm
Location: West Orange, NJ

Postby hein on Thu Oct 02, 2003 9:45 pm

jason wrote:The easiest way to force a reload of a particular channel is to bring that channel's configuration up within the Administration interface, and to click on the 'Update' button. This will cause the channel to be repolled on the next iteration of the channel poller (within 15-30 seconds).


This fails to work when nntp//rss talks to a proxy server that thinks that the cached file is still up to date :-(
hein
 
Posts: 5
Joined: Tue Sep 30, 2003 8:00 am

Postby jason on Tue Oct 07, 2003 1:19 pm

Good point - I don't believe there is any way that nntp//rss can force the proxy server to refresh its cache. I'm guessing that the original RSS feed has some interesting expiration headers that is making the caching proxy server keep a copy of the RSS document past its 'freshness' date.

Which RSS feed is causing difficulties?
jason
Site Admin
 
Posts: 114
Joined: Sat May 03, 2003 10:44 pm
Location: West Orange, NJ

Postby jvinocur on Sun Oct 19, 2003 2:15 pm

jason wrote:Good point - I don't believe there is any way that nntp//rss can force the proxy server to refresh its cache.

Sure there is -- check section 14.9.4 of RFC 2616 (HTTP/1.1).
jvinocur
 
Posts: 3
Joined: Wed Aug 27, 2003 12:57 am

Postby jason on Sun Oct 19, 2003 2:56 pm

Jeff - Thanks for the heads-up - I wasn't aware that the HTTP spec allowed for clients to force a non-cached retrieval. I'll look at a suitable implementation for this in the next release.
jason
Site Admin
 
Posts: 114
Joined: Sat May 03, 2003 10:44 pm
Location: West Orange, NJ

Postby peterj on Tue Oct 21, 2003 5:31 pm

I had exactly this problem, and resolved it with a judiciously-placed
Code: Select all
setRequestHeader("Pragma","no-cache");
in org/methodize/nntprss/feed/Channel.java. (I can post a diff-format patch if you want, but it's a simple one-liner.) Since you're using If-None-Match and If-Modified-Since anyway, it seems appropriate to override the proxy's (broken? misconfigured?) caching.
peterj
 
Posts: 9
Joined: Tue Oct 21, 2003 5:11 pm

Postby jason on Tue Oct 21, 2003 5:48 pm

Thanks - I think what I'll do for 0.4 is make this a configurable option within the Proxy section of System Configuration. That way, for users with working proxy servers, they will not force a non-cached retrieval for every document.

For those users with misconfigured proxy servers, or feeds returning inappropriate caching headers, they'll be able to check an option which will include the Pragma header within the request.

A question to those of you watching this thread - is a system-wide level of granularity for this setting good enough? Or do you have a specific scenario where you'd like to be able to set this at the channel level? i.e. You have certain feeds that are returning invalid headers causing the proxy server to cache those documents for longer than expected.
jason
Site Admin
 
Posts: 114
Joined: Sat May 03, 2003 10:44 pm
Location: West Orange, NJ


Return to Feature Requests

Who is online

Users browsing this forum: No registered users and 1 guest

cron