NOTE: This documentation is not yet finished. It's included here as it may be useful in its current state; it is likely to be finished and checked by around April 10, 2006. - Joel Burton ================================================== What People Need to Know About CacheFu By Audience ================================================== CacheFu is a set of technologies meant to allow Plone sites to perform significantly faster, while still showing content and navigation that is up-to-date with the changes being made by content editors. It does this by combining several existing products and adding a lot of configuration controls. With few or no changes, most sites will experience significant performance increases, and, with tuning, many sites can experience even more significant increases. -------------------------- An Introduction to Caching -------------------------- A smart program like Plone will always be a bit slow. This isn't an oversight on the part of the developers, but is a side-effect of all the work that Plone does for you when it gets a request for a page. Imaging a user requesting a page, ``/newsitems-project-a``, from your site. Plone has to: - Check if this user is logged in, and, if so, if their login credentials are good (remember, the web is a "stateless" protocol--each request for a page or an image is essentially a separate thing, and must be considered and checked independently). - Find the object in the object database. - Check that the user has rights to that object (and to everything else mentioned here, like the skins, scripts, etc.) - "Skin" the image by applying page templates to it. - Perform any logic required (for example, the portlet on the right that shows the calendar must search across all events to decide which ones are in the current month, and show them on the calendar). - Add custom information about the user (user bar links, their name, preferences, etc.) - Decide which JavaScript and CSS files to reference. - Create dynamic left-hand navigation, global section tabs across the top, etc. - Figure out which portlets to show, and show them. - and more! This is a good thing--all this checking and dynamic stuff is what makes Plone rich and easy-to-use. However, it means that, out of the box, Plone can only serve a few web pages in a second on most servers. For lightly-trafficked sites, this may be acceptable, but for more popular sites, this means that users may have to wait a second or more to get back their page, making for a slow experience. One excellent solution to this is caching--having Zope/Plone and other programs work together to decide which parts of that workload don't need to be performed every time, and use the old results (for those that aren't used to the idea, a real-world cognate is your friends' phone numbers: you could look them up in the phone book *every time* before you call them, but, to make things more faster and more pleasant, you only do that if you know it changed or if you've forgotten what the number was). Done correctly, caching is a tremendous benefit--the site runs much better and no one notices anything different. Done incorrectly, users become frustrated that things don't change as expected--they add a news item, and, moments later, they visit the folder where they added this, and the news item isn't shown in the contents listing. This is a "stale" result-- the user experiencing something old. Sometimes, some degree of staleness is an acceptable trade-off. For example, on a site with a portlet that shows the weather, you probably don't want to contact the government weather server 600 times a minute to see if the weather forecast really changed. Instead, you compromise by checking only every 5 minutes (or 5 hours or whatever); users can get slightly older weather reports, but it's a worthwhile trade-off for performance. Out of the box, CacheFu does *not* make these compromises for you, except in one or two very small areas. You can trust that it works hard to get you the most up-to-date content it can, even if it means your server might be doing a bit more work that is the possible low-point. For those who want more control, this document will help them understand how to turn those knobs. --------------------------- Where Things Can Get Cached --------------------------- Essentially, there are three broad places web pages or parts of web pages can be cached: - **Inside Zope**. This data is cached inside Zope, meaning that Zope still creates and serves the web page directly, but it may not have to perform some time-consuming calculations every time. Generally, inside-Zope caching has the least dramatic speed benefits, but when content changes, it is easy for Zope/Plone to realize this and begin serving new content. - **On a proxy server / caching appliance**. These type of programs stand between the web browser and Zope. When the user, Jane, requests a web page, her request actually goes to the proxy server. This server decides whether or not to pester Zope to produce the page or image. In many cases, the proxy server has this information already, and serves it to Jane. This is much faster, since proxy servers tend to be heavily optimized for this operation. In cases where the proxy server can't serve the content directly to Jane, it will pass that request to Zope. In some cases, the proxy server will be told that, in the future, it can serve this directly, and it was cache this response, and will handle it directly in the future. Squid is a very popular, open source proxy server, and CacheFu is optimized to work with Squid. Apache, while being a more general, very extensible web server, can also handle caching. Apache does not have the features that Squid has, though, for "purging"--getting rid of content that is no longer as up-to-date as it could be. Therefore, Apache can be used to proxy cache for certain things, but CacheFu will not be able to use it to as much of an advantage as Squid. It is advantageous, then, for medium- and large-traffic-sites to use Squid. Smaller sites will get some benefit, too, but these benefits may be less required, and may not justify the addition of a new piece of software. - **In a web browser**. Almost all web browsers (and all the well-known, popular ones, such as Internet Explorer, Firefox, and Safari) can cache web pages and images themselves. When they do this, they are able to respond themselves to repeated requests for that same image and page. This is obvious incredibly fast--since the request and web response doesn't even have to go across the network at all, and, in some cases, neither Zope nor a proxy server will have any idea that there was even a request for it. The general challenge with browser-side caching is that there's no mechanism to let every possible web browser know that they should let go of those pages they've been holding onto (it's not feasible to try to send out a purge request to everyone that visited your site, and practically no browsers even support this feature, anyway). As such, the kind of content that CacheFu caches this way will tend to be things that it knows the browser can cache without any risk of that content becoming stale. In addition, there are four other common places where caching might happen in the Zope/Plone world: - **In a ZSQL method or in a database**. In cases where you have connected Plone to a relational database, you can tell your ZSQLMethod (the component of Plone that is an SQL request) to cache the result of that database request. So, for instance, in an application that lists stock prices, you could opt to only query the database once every 5 minutes, even if the SQL method was called every second. Similarly, some databases themselves have their own mechanisms for caching requests and responses; Zope is not involved or even aware of these. - **In authentication against LDAP/Relational Databases/etc.**. Plone can authenticate users against special storages for user data, most commonly LDAP servers and relational databases. There are different add-on products for these different kinds of servers. These add-on products sometimes include features that cache the authentication itself--that is, they don't try to memorize the result of getting a page, or what an image looks like, but they cache the fact that "joelburton" *is* a user of your system, that the password "jUnER0x!" is my password, and that I am a Manager on your system. Rather than pestering the LDAP server on _every_ request, the software caches this information, and trusts that I haven't been fired in the last 5 minutes from your staff. - **ZODB caching**. The Zope Object Database (which normally holds all the Plone content) itself caches those objects. In other words, when someone requests to see the content item at ``/news-items/project-a``, the ZODB will hold on to that for a few minutes, and, for further requests, will just use the in-memory copy. This is just the "raw" object--not the result of calling it, skinning it, or anything else. This kind of caching is "transparent"; it is never stale, and is never a bad thing to have. - **ZEO caching**. ZEO is a technology for scaling large Plone sites across multiple servers, by separating the object storage onto one server, and the application server onto other(s) (though they can both be on the same server, and other configurations are possible). ZEO itself will tell the individual application servers that it's ok to cache requests for an object (again, /news-item/project-a), so that, if that same Zope application server needs it, it can use it's private copy. This caching is very similar to the ZODB caching, above (it's essentially a networked copy of the same idea), and has the same transparent, always-beneficial characteristics. None of these four types of caching are used or affected by CacheFu. The latter two are perfectly transparent and never affect anything. The first two are very application specific--they're turned on by programmers and site-builders specifically. CacheFu won't disable these or enable these itself, and the caching CacheFu provides will be *on top of* whatever benefits these provide. ----------------- Primary Audiences ----------------- CacheFu, and caching in general, can be quite complex--for those who want or need to understand this complexity. Rather than trying to teach everything to everybody, this section teaches the concepts for different audiences. People building sites in Plone need to know less than people who build complex products for others to use; people who deploy ordinary sites need to know less than people who deploy enormous sites with thousands of users or more. End Users ========= End users of the site (the people who use the site, but don't create content or design the skins, etc.) need to know nothing about CacheFu, nor do they need to make any changes in their browsers. CacheFu already sends out the right commands (caching headers, as they're called) to say things like "don't cache this, even if the browser normally wants to cache things". .. tip:: With the standard install, there's only one case where users (only anonymous users) might see stale content. When a user is looking at a page (say, ``news-items/project-a``, Plone will normally show navigation on the left-hand side that might include "sibling items"--other items in the ``news-items`` folder. *If* you are using Squid, it will serve cached pages to anonymous users, and the cached copy of ``project-a`` might show navigation that doesn't include the newly-created ``project-b`` news item. This only happens with anonymous users, and only when Squid is involved (because CacheFu tries to cache things most aggressively in these cases). This case "times out" after an hour--even if nothing else changes, CacheFu has told Squid to not serve ``project-a`` for more than an hour without re-checking, and so it will, at the very latest, pick up the new ``project-a`` page showing ``project-b`` in the left-hand navigation within an hour. If you can't tolerate this experience, you could reduce the window of time to less than an hour, or, if you absolutely need 100% up-to-date navigation for sibling items, you can turn off Squid caching for this case, (covered later). For logged-in users, there are no stale content opportunities with the standard settings: non-anonymous users never have their content pages cached in Squid by CacheFu's default settings. Content Managers ================ For the people who edit content, there's nothing they need to know, except for the warning about stale sibling items in "End Users", above. The content managers themselves, won't experience this, but the users who use their content might. If a content creator sends a link to newly-created content, everyone will be able to get to it (assuming, of course, they have the right permission to do so). However, should a content manager send out a link to ``news-items/project-a``, that page might not show the even-more-newly-created ``project-b`` in the navigation for anonymous users, as described above. ZMI Customizers =============== This audience makes up the bulk of people who build Plone sites and customize them. It includes people who do things like: - customize templates and CSS - write Python Scripts and use External Methods - make setting/configuration changes, like changing how the navigation displays, etc. It's important for you to understand that you don't have to change *anything* to get many of the benefits of CacheFu, but, as you write new portlets, or skins, you may need to adjust the settings in the product to continue to not get stale pages. First, let's look at the general configuration for CacheFu. Cache Configuration Tool ------------------------ Most of the settings for CacheFu's technologies are set in the "Cache Configuration Tool"; found in `Site Setup -> Cache Configuration Tool`. This tool has four tabs: - "Cache configuration tool". This is where broad configurations to the behavior are made. - "Caching rules". This tab allows you to determine what kind of caching is chosen for different situations. - "Caching header sets". This tab allows you to decide exactly how the caching rules get carried out. - "Cached macros". This tab is for the experimental macro caching feature (not enabled by default). - "Page cache". This allows you to clear the in-memory storage of some kinds of cached content. All of these are described below in more detail. Cache Configuration Tool Tab ++++++++++++++++++++++++++++ Cache configuration CacheFu can be used on a site that is "Zope only". This means that no proxy server (either Apache or Squid or anything else) is sitting in front of Zope. In this case, things can be cached in Zope, or in a web browser. If, however, you put your Zope behind Apache (or any other non-Squid caching proxy), CacheFu can send out headers to tell the proxy server to cache JavaScript, CSS, etc. If you're behind Squid, CacheFu can send out headers to cache those things as well. Plus, since Squid support "purge requests" (to let go of cached content and get it fresh), CacheFu can also send out headers to tell Squid to cache many more pages, since it can selectively clear those from Squid's cache as needed. In some cases, your organization may want or need to use Apache (as a very full-featured web server, it can do things that Squid can't, and has dozens of add-on products). However, you may still want to get the benefit of Squid. This is a case to use the "Zope behind Squid behind Apache" setting: the public talks to your Apache server (which might also handle things like PHP applications). Requests for anything on the Plone site are delegated to Squid, which can either respond itself or further delegate to the actual Zope/Plone server. If you are going to run with Squid, either by itself or behind Apache, be sure to read the "Setup with Squid" section, below, for important information on how to configure Squid. If you are going to run with Apache, you will need to have a few settings in your httpd.conf virtual host block -- see below. If you are running your Zope site by itself, there are no special configurations required. Site Domains In order for CacheFu to be able to tell Squid to purge a cached page, CacheFu needs to know the domains that Squid might have that page under. For example, if you're serving `www.example.com`, a cached page, `about-us` could be at http://www.example.com/about-us. Depending on your Squid (or Squid+Apache settings), people might also be able to visit http://example.com/about-us (note the missing "www") and find the same page. CacheFu will need to tell Squid separately to purge both of those URLs; while CacheFu understands that this is the same page, Squid has no idea that these two pages are the same thing, and must be told separately. Therefore, you'll want to list **all** of the domain names that your site is reachable at. Be sure put the port number at the end, even for port 80, and don't forget to include `https://example.com:443` and `https://www.example.com:443` if you run on HTTPS, too. Please note that port numbers here are *the ones that the public visits*, not where your Zope instance is really speaking. Most Zope servers serve content themselves on port 8080 (8282 is common on Mac OSX); however, as far as Squid is concerned that content was asked for on port 80, since that was the original request. If you're not using Squid, you can keep this empty. Values entered here are only meaningful if you chose "Squid" or "Squid behind apache" above. Squid URLs If you're running Squid and Zope alone, Squid normally answers web requests from the outside world on port 80. Therefore, CacheFu knows how to reach Squid (on port 80) to send purge requests. You can leave this blank. In some cases, you're not running Squid on port 80. Most likely this is because you're running Squid behind Apache, and you run Apache on port 80 so it is the first server to handle the request). In this case, you need to tell CacheFu how to reach Squid to send purge requests. Normally, this will be `http://127.0.0.1:3128`, the address and port number to reach Squid on the local box. If you are running Squid on a different box, or on a different port number, you'll want to enter that instead. If you have several Squid instances, you'll want to list *all* of them so that each purge request can be sent to each one. Compression Separate from caching, CacheFu can also compress web pages. This compression is a standard part of the HTTP/web technologies: pages can be sent compressed with "gzip compression", and most web browser can receive the compressed page, uncompress them, and render them from the user. You can choose to "Never" do this, which is the safest option--you won't have any browser incompatibility issues to worry about. You can choose to "Always" do this. This is an **unusual choice, and probably not correct**--some browsers can't deal with gzipped pages, and always sending them will not allow people with these browsers to use your site. You can have CacheFu decide on a case-by-case basis depending on what the web browser sent for the "Accept-Encoding" header. This header is sent by web browsers to indicate what kind of content that can receive back. If the browser indicated that it can receive gzipped content, this will send it gzipped. You can have CacheFu decide based on both the "Accept-Encoding" header and the "User-Agent" header. "User agent" refers to which web browser the browser says that is. If this option is selected, CacheFu looks both to see if the web browser *says* if can handle gzipped content, and, just to be safe, it checks that it is a browser that CacheFu knows can do this successfully (some early versions of Netscape are buggy for this!). Checking the user agent comes at a high cost: squid will need to cache separate versions of each of your pages for every single browser / operating system combination, which will make cache hits much less likely and will increase the disk space required by squid by a factor of 20-100! Using Accept-Encoding is the recommended practice, since the buggy browsers are rarely in use. Vary Header In order to not serve stale content, cache systems (either proxy caches like Squid or in-browser caches) need to know more than just the URL of the object. Returning a cached copy of `http://example.com/about-us`, for example, might be wrong if the user prefers to speak Greek, and the cached copy is the English version. Therefore, in this field, you can list all of the values that are understood by browser/cache proxies for "varying" these results. For example, if the Vary headers just "Accept-Language", and a request comes for `http://example.com/about-us`, your cache program will cache it *while keeping track of the Accept-Language value it received from the browser*. Then, when another request comes, it will make sure to hand back the cached copy only if the Accept-Language value for the new request is the same as the old. Otherwise, it will cache this second copy, and hand that back only for the same URL and Accept-Language header. If your site has multilingual capabilities, you'll want "Accept-Language" in here. This will make sure you don't return the English copy for the Greek speaker. If you don't have any multilingual content, and you don't even want the standard Plone templates to be returned in other languages (which you can prevent by removing PlacelessTranslationService from the products), you should remove this. Keeping this value in would mean that your caching systems are keeping a separate copy of a page for Accept-Language=en (English speakers) and a different copy for Accept-Language=gr (Greek speakers) *even though* the pages themselves don't vary based on language. Should you fail to remove this, you will use up more memory for caching (and have an occasional request go to Zope that could have been answered from caching) more often than is necessary. If you allow compression (see above), you'll want to add "Accept-Encoding" here. This will ensure that you don't consider a gzipped version of a page and a non-gzipped-version to be the same thing. In other words, we'll cache two copies of each page: one for people who say that can accept gzip, and others for those who can't. If you have a multilingual site *and* allow gzipping, you'll want to leave both in. Caching Rules +++++++++++++ Caching rules are part of the core concepts of CacheFu. They are a set of rules which are analyzed for each web request that gets to Zope, and, if the rule matches the request, the caching rule puts its behavior into effect. For example, if a web browser gets some CSS, and this request gets to Zope (i.e., it isn't answered by Squid or other places), the request goes through the rules here, in order (top to bottom) until it matches one rule. That rule that matches can do things like: - Cache the page in memory in the ZODB - Request that headers be sent out with the response. These headers are interpreted by browsers and cache proxies to say things like "keep this in your cache for 1 hour" or "never, ever cache this" (for some details on headers, see the next section). Since we take the first rule that matches, only one can ever apply to any given request. If no rule matches, no particular action is taken, and the content is still rendered and returned normally. Let's walk through the rules that CacheFu ships with. Understanding them will help you understand what CacheFu does. HTTPCache This cache is never matched (FIXME: not sure what this means?) out-of-the-box for CacheFu, but, if you customize things, can be quite useful. It is used if there are places where you have pages (or images, such as your site's logo) that can be *entirely* cached and do not require any invalidation. Consider a site with a page that _always_ looks the same, regardless of whether you're logged in or not, and which doesn't need to be purged when any content changes. A good example might be a page like that pops up with static help about your site, and doesn't show content, rely on content, or change based on your login status. For this example, that page would be a Page Template called `site-help`. To tell CacheFu that this page can be cached like this, you'd need to associate it with the "HTTPCache" cache manager. For example, we could go to our `site-help` template and, under the Cache tab, associate it with the HTTPCache. Traditionally, to do this in Zope, you'd be associating the PageTemplate with a cache called "HTTPCache" which is an "Accelerated HTTPCache Manager", and that manager sets the headers itself. In CacheFu, however, HTTPCache doesn't do anything itself--it's essentially a "marker" to indicate that a piece of content *can* be cached like this, and therefore, it's picked up by this rule. Since no content types are selected in the box below, all types of content work can meet this rule (assuming, of course, they're associated with the HTTPCache). Several options aren't used here (& will be discussed where they are used, later). The two boxes for headings for anonymous users and headings for authenticated users are the primary "outcomes" for rules, and this is the case here: content that meets this rule is, by default, matched to the cache-in-browser-for-24-hours rule, both for anonymous and logged-in users. If you wanted to cache this stuff just for 1 hour, you could change to the cache-in-browser-for-1-hour rule. If you wanted a different possibility (say, cache-in-browser-for-10-mins), you'd have to create this in the caching header sets tab, described below. The last choice, "Last-Modified Expression" is the expression that will be used to decide when this content was last modified; this is used so that CacheFu only caches it for 24 hours beyond that (or one hour, or however long you chose). Content This rule is used to cache displays of normal (non-folder, non-image content), like a News Item. By default, the normal Plone content types are cached by this; you don't have to associate them with any cache manager, like we did above. The content types that are matched by this are selected in the content types box. These are those content types that aren't File or Image (those are handled separately) or folderish thing (those are also handled separately). Of course, we don't really want to cache the news item *itself*-- that's not sent to web browsers. Instead, we want to cache the HTML view of a news item. The "Default view" box, when checked, means that this rule will be in affect when a request is made for a skin object (Page Template, Python Script, etc.) for the content types listed *if* that skin object is the default view for that content type. So, for example, it will catch:: http://news-items/project-a and :: http://news-items/project-a/newsitem_view since ``newsitem_view`` is the default view for news items. It won't catch :: http://news-items/project-a/special_newsitem_view Should you want to cache additional views, like ``special_newsitem_view``, you should add that to the "Templates" field, which is for the ids of additional skins that should be cached. "Cache Templates" in memory means that, in addition to whatever else kind of outcome we would have for things that match this rule, we should cache the template results in memory. This helps so that, even if you have no proxy cache, CacheFu can still work with the in-memory cache. Note that :: http://news-item/project-a and :: http://news-item/project-a?form_var=1 are different requests, and, as such, will be cached separately. "Cache Preventing Request Values" is used to specify those form variables (or other things in the request object) that, if present, should signal that this request should not be cached. By default, "portal_status_message" is listed here. "portal_status_message" is the request variable used to hold the feedback messages (usually shown in bright orange, at the top of pages). Since the same template can be shown with dozens of these feedback messages ("Changes saved", "Content added", "Email sent", etc.), it would be expensive to cache each copy individually, and probably not that helpful, since not many people would want to see the exact same page, anyway. Therefore, if there is an status message being sent, CacheFu won't cache this page. "Predicate" lets us add any arbitrary TALES expression as a further check for whether this rule matches. A request would have a) be skinning a content type listed in the content types, b) be for a default view (if that's checked) *or* be an explicitly-named template, c) not have a `portal_status_message` and d) pass this expression to match. "Header Set for Anonymous Users" is set to have the proxy cache (Squid, usually) receive the message to cache this for 1h. For logged-in users, it's cached with an ETag, a mechanism that allows it to be cached in ways that cause changes to request a fresh copy. .. tip:: ETags ETags are a mechanism that is a HTTP/web standard for web browsers and servers to pass around some information that represents "other information" about a request. This information can include things like: the kind of user that requested it, the date it was last changed, etc. So, if you request :: http://news-items/project-a the server might give this back to you with an ETag of :: ETag: joelburton-2006/12/25-en Which means that this request was for Joel Burton, the content was modified on Christmas, 2006, and it was in English. The cache program (be it in the browser of cache proxy) will store this ETag along with the content. (Note: this isn't the exact format that ETags appear in, but the concept is accurate). Now, the browser wants to re-request this page from Zope, it will get that old ETag and give it to the Zope. Zope will look at it and compare it with the current ETag it would create. If they are the same, Zope will return a HTTP response that "nothing has changed, feel free to use your current copy". If they differ, it will return the new page. So, for example, if Joel re-visits http://news-items/project-a when he is logged out, his browser will hand Zope the ETag it has from his last visit, joelburton-2006/12/25-en. Zope looks at the logged in user (now Anonymous) and creates a new ETag:: ETag: anonymous-2006/12/25-en (requested by Anonymous, content last modified on Christmas, prefer it in English). Since this is different than the ETag that the proxy cache submitted, Zope will return the content itself, since anonymous users might need a different page (without Joel's name, for instance, or with different things showing because of security). Squid cannot handle requests with ETags, since it has no way of knowing who is logged in, when content was created, and so on, so all of these requests get routed around squid. The "ETag Components" choice allows you to select what things should make up the ETag; as you choose things here, if any of these things changes, it would cause the ETag to change, and cause the page to be recalculated. The default settings are sane but cautious: it causes a re-examination if there are changes to the user who requested the page, the current skinpath (i.e.,, Plone Default, Plone Tableless, or a custom skinpath), whether they want gzipped content or not, and the time of the last catalog change. The list is an interesting idea. If you edit a news item, of course, we'd want CacheFu to stop returning the old copy of that news item view. However, if you edit the title of the folder that news item was in, we'd *also* want CacheFu to stop returning the old copy of that news item view, since the old title of the parent folder would appear in the navigation portlet, breadcrumbs, etc. By having the date/time of the last catalog change factor into the ETag, *any* change to *any* object will cause CacheFu to re-create the content pages. This guarantees fresh navigation, but at the expense of re-creating things where staleness might have never been an issue, or where it might have been acceptable. This ETag setting is just for logged-in users; for anonymous users, remember, the page is cached in the proxy for an hour regardless of ETags, and, as such, any change to the site won't cause everything to be unusable from cache. If you don't care about this, and desire more performance, you could opt to change the "Time of last catalog change" to "Context modification time". This would mean that editing anything won't invalidate all ETags, but it does mean that an old title for a sibling might appear in the navigation tree or breadcrumbs, or such. This can significantly speed up performance, and may be worth considering. In addition to the checkbox ETag choices, you also can indicate which request values should be factored into the ETag. The default is month, year, orig_query. This is used for the calendar portlet--you might have changed nothing on the site since this page was requested a while ago *but* this request is for a user who has clicked the next-month button on the calendar, and, as such, needs a different response: the same piece of content, but with the calendar portlet showing the next month. If you add other form variables to your view templates and changes to these should cause the page to be cached differently, you should add them here. Note the difference between the "Cache Preventing Request Values", above (which held "portal_status_message") and this. The cache-preventing-values piece says that *if this value exists, don't match the rule* (and therefore, don't send out the caching headers, etc., for this rule). The "ETag request values" field says *you can cache it if this value exists, but consider it a different request*. "ETag Timeout" is a failsafe for ETags. This is the number of seconds that an ETag should ever last. At 3600 (the default here), it means that *even if* the ETag comparison suggests nothing has changed, that Zope should generate a new page anyway. It's good to have this--this way, if you forget to have your caching predicated on a certain value, you *still* won't hold onto stale content views for more than one hour. "ETag Expression" allows you calculate anything as an additional ETag value. "Purge Expression" is a script that can handle the purging in Squid of content views once changed. You shouldn't have to edit this. Containers This rule is used for folderish content types (Folder and Large Plone Folder). It is similar to the Content rule. The difference is that, for anonymous users, the Content rule caches the view of the content object until the object changes, whereas, that would provide too much stale content for views of folderish objects. The view of a folder traditionally is a list of the child objects, so when a folder gets a new document added to it, or has one of its child documents edited, the view of the entire folder needs to be changed. It's difficult to decide if any child object was changed; instead, CacheFu relies on the quick (but much more general) rule of: has any object in the entire catalog been changed? If so, the view for a Folderish object must be regenerated. Therefore, these are never cached in Squid or Apache (it would be difficult to purge every possible view of every folderish content item on every catalog change!) and are cached in Zope, using the same ETag strategy as for the Content rule, above. Essentially, both anonymous and logged-in users who view container views are treated the same as logged-in users viewing content with the Content rule. Templates This rule is for "templates"--skin objects like Page Templates that are not presenting a piece of content, and are not forms. A good example is the "accessibility-info" template that comes with Plone; this isn't content, is not a form, but is a page that could be cached until it changes. The "templates" field is a list of those templates that match this rule. As you add new templates that could be matched by the rule, you can list them here. These are cached with ETags, using the same strategy as for Content and Containers rules, above. Similar to the Container rule strategy, the cache is cleared for these every time the catalog changes. Almost all template rely on the main template, which relies on a dozen other templates, and it would be very difficult to track all of these dependencies. Therefore, any change to anything in Zope might affect what these template should show, and, as such, clears them. This is safe and conservative. Some sites may decide that it is acceptable, for anonymous users, to cache these Templates in a proxy server, so that they are cached for an hour (or 24 hours, or more). Changes to the template itself, or to the `main_template` or other template won't appear, but these sort of changes might be very rare on a production site, and, if they happened, the sysadmin could restart the proxy server to clear it's entire cache. .. tip:: Why Not Forms? So, why aren't forms cachable with the Template rule? Forms are use in Plone both to show the form, and also to re-present the form with validation errors; if the form was cached and used from cache, it might have validation errors and already-entered data from a previous user. FIXME: geoffd, couldn't this be fixed by adding form.submitted to the 'Cache preventing request values' box? Or are there other reasons why forms shouldn't be cached? [GD: yes, probably, but I don't think it would be a huge win] XXXX END OF WRITTEN PART - More likely to use Apache than Squid, or to use nothing at all - But everything should "work" w/o Squid, even if things are cache-in-mem rather than cache-in-squid when they could be - Should know to associate w/HTTP Cache for fine-to-cache full pages (ie, homepage if w/o personalization & portlets) - Cloned content types that are non-folderish should be added to content rules - New views for content (ie, news_item_short) that aren't on the display menu choices should be added to content tab - If add portlets that use form vars (ie, show-more-news-in-portlet or show-less-news-in-portlet), add this formvar to "etag request item" in all places - Cloned folderish types should get added to the containers tab - New views for folderish things that aren't on display menu choice should be added to containers tab - New templates that aren't contentish should be added to "templates" as long as it doesn't depend on things like clock - if template uses formvars, add them to cache_preventing_Req_items, unless they're commonly-shared (worse case is that we cache too many choices) - Nothing to know about for CSS/JS - If we clone new File/Image types, add to file/image - Know when to clear page cache [XXX: when is that? GD: for debugging] - *Might* be helpful for them to understand the header-settings, but only the most basic (ie, cache-for-1h-not-1d, etc.) - XXX: should we ship additional header settings (ie, ship with "cache-in-mem-for-1h" v "cache-in-member-for-1d" (both of which could still be cleared, of course, but would have diff max cache lengths) - RAMCaches still work exactly the same, so you can still do things like cache production of news-item-portlet-search - Nothing here to clear those RAM caches in advance, or in any smart way - RelDB caches still work the same - Nothing here to clear those reldb caches in advance Intermediate Developers ======================= - [These are people who are building Archetypes, building web apps, but not neccessarily hard-core-geeks] - FIXME Advanced Developers =================== - [These are people who are seriously customizing Plone, and will learn more knobs to get more power] - FIXME System Admins ============= - [People who administer the Squid/Apache stuff, but aren't necc Zope/Plone people] - FIXME ----------------- Special Audiences ----------------- People may fit into more than one of these, in addition to above audiences. People who store uesrs in LDAP / Rel DB ======================================= - Default caching is too aggressive to cover this case - Should check "user roles" everywhere that "user id" is checked in ETags - Covers case of "joel was manager but changed in LDAP to make him just member" - Since that won't change catalog, and therefore won't work w/existing etag choices People who don't store content in ZODB ====================================== - (ie, storing content on filesytem or in relational database) - If catalog doesn't change on content change, many clear-cache features won't work - Default caching will be too aggressive - [FIXME: Is there a built-in fix for this? They could just not add those fs- or reldb-stored content types to "content" tab, right? This miss anything significant?] People Who Want to Learn to Use Squid ===================================== - FIXME: What do they need to know to do this? LDAP/RelDB storage of users =========================== - Cachefu can act on user roles or on user, and not know that user perms have changed