Well a short answer, we can not and should not

• server-side cache (=joomla cache) is INDEXED programmatically by using URL and PHP logic
  ... and it can return different pages for the same URL

• browser cache is INDEXED by URL,
  ... and you can not have different pages for the same URL,
  (unless you use AJAX call for all/most content, when partly defeats the idea of browser caching)

(Longer answer)

Problems if not done properly:

• it is a bit bad for the user to see his unlogged (guest) page for a URL, or his logged page after logout
• and gets even worse if by mistake we instruct a proxy to cache the page meant for a specific logged user !! which a security problem, as all users visiting the URL will see the cached page, get his/her cookies

The URLs do not change, but the user state does change

• login / logout
• other 3rd party extensions (modules, plugins) that can even do per guest content using cookies, etc

so you can enable browser caching for a URL only if you know that page will not be different

• e.g. not different after login,
• and if you also know that other installed extensions e.g. modules, will not display different content too (or if you allow the site-maker to say this, and enable browser caching via parameter, because he/she knows that he/she does not have such modules or it is ok to show stale module content for some period, as long as component is ok)

Joomla after login/logout will serve a different (cached) page

• because component is written properly to examine the URL parameters and then also execute other logic and add some extra URL parameters, which it will use to index the Joomla cache Index (sorry if my terminology is wrong)

But if you instruct the browser to cache the page, the above will have no effect because the browser will not re-request the page because URL is the same

• a common practice when you want to enable browser caching is to make the URLs unique by adding an extra URL variable to the URLs and then enable the browser caching (but not the proxy caching)
  (which is enough to avoid must re-validate/re-submit warning message in search forms when clicking back/forward)
  the URL parameter will be added to the form target via JS that manipulates the form

• then you can make it even more nice, and detect login from a different browser window / browser TAB

• and then if you want it better, then also set a cookie (set it inside onAfterRender event) that will indicate if a page is good or stale (e.g. login/logout), thus when user click back or forward, then javascript will force the page to be redownloaded, avoiding the user seeing the stale content

A very good reading that explains how headers should be set
(does not include programming it, just gives information how to set headers to avoid problems):

https://code.google.com/p/doctype-mirror/wiki/ArticleHttpCaching

• about proxy caching we should avoid it always, if we serve content for logged users, because in the case of mistake (and we do make mistakes) it can be bad
• also always set expires header to the past and instead use max-age=300 JResponse::setHeader('Cache-Control', 'private max-age=300'); // no proxy caching with 5 minutes browser cache to the future according to user's time

i say the above about "expires" header, because if it happens (ever)

• that the server date is wrong and you use it to decide an expires header
• or you make an error in your code, then you may instruct the browsers of all your visitors to cache the page for years

it is a bit bad for the user to see his unlogged (guest) page for a URL, or his logged page after logout

Let me be clear about what the problem is; I am experimenting with Varnish at the moment . The idea is to have pages cached when an unregistered user (which is the majority of visitors) is visiting website. In that case the content is always the same and there is no reason for sending no-cache header.
As far I can say Drupal and Varnish are working quite good together.

Cookies are not issue because as far there is a cookie sent by the backend server Varnish won't cache the URL
On the other hand cookies are an issue because Joomla! is sending always a cookie even if this is not necessary. But one step after another.

about proxy caching we should avoid it always

Well I completely disagree on that. Every bigger website is using it because this exactly the way how you handle high traffic properly. But as far Joomla! is basically not able to work together with e.g. Varnish which may be a factor to choose other CMS over Joomla! by serious clients.

Regards,
Radek

about what you are trying to do, yes i understand the usefulness that you describe

• about proxy caching, there is a misunderstanding
  i did not say "always"

• i said always if serving content of logged users, and this is not my personal saying / opinion, it is a general guideline

even if you make sure that the URL of a logged user is unique (e.g. some hash value in the URL), and then you instruct proxy to cache it, then someone learning this URL in someway, can request the page through the proxy and retrieve private user pages and cookies

• you can enable proxy caching when a URL is only for guests and logged users will have a different URL that proxy will be instructed not to cache it

• I am not sure browser caching functionality can be made easier by adding something -more- to the CMS configuration or API

even if you make sure that the URL of a logged user is unique (e.g. some hash value in the URL), and then you instruct proxy to cache it, then someone learning this URL in someway, can request the page through the proxy and retrieve private user pages and cookies

I didn't say that. As I wrote before, what I am trying to achieve is to chase only if the user is not logged in. So there is no security concerns whatsoever

you can enable proxy caching when a URL is only for guests and logged will have a different URL that proxy will be instructed not to cache it

No. It is not necessary. For now I can differentiate between logged in user or visitor. Let me try to explain it step by step: Let imagine a user is visiting the main URL "/"

• User is logged in: Varnish determines that the user is logged in and call the backend server (Apache?) directly
• User is a visitor and there is no cached version of the URL: Varnish is caching the URL for further use

Next round

• User is logged in: Varnish determines that the user is logged in and call the backend server (Apache?) directly
• User is a visitor and there is a cached version of the URL: Varnish returning the cached version of the URL instead of calling Apache

There are basically 3 things that preventing that scenario right now:

• Varnish do not know if user is logged in or not. This is however very easy to achieve by sending a custom header along with the response. This is how I implemented this so far
• Joomla! is sending always no-cache header
• Joomla! is always generating a session cookie even of it is not necessary

The first and second issue are really easy to solve. The biggest pain would be the cookie thing

Joomla! is sending always no-cache header
Joomla! is always generating a session cookie even of it is not necessary

ok , yes i see, what you mean

• just we do need a default header, that will work in ALL cases, and it seems that default header does (maybe a configuration option to disable default header)
• about session cookie for unlogged users, the unlogged users also have a session so i am not sure if that can be avoided (can it ?) So you cannot configure varnish server to decide by its own by looking if cookies exists, instead it must rely on the Joomla component to use different URLs for logged and also set 'cache-control' HEADER properly

Here is what i did inside 'onAfterRender' event of a system plugin, to change default behaviour

$jinput = JFactory::getApplication()->input;
$option = $jinput->get('option');

// $browser_cachable, was DECIDED and SET by component display() controller task
// 0:  no cache
// 1:  public content (unlogged user)
// 2:  private content (logged user)
// null default joomla header behaviour
$browser_cachable = $jinput->get('browser_cachable', null);

JResponse::allowCache($browser_cachable ? true : false);
JResponse::setHeader('Pragma', $browser_cachable ? '' :'no-cache');

// 1:  public content (unlogged user),   2:  private content (logged user)
// 0: not cacheable, (so it is best to set to private)
$cacheControl  = $browser_cachable == 1 ? 'public' : 'private';

// SET MAX-AGE, to allow modern browsers to cache the page, despite expires header in the past
$cacheControl .= ', max-age=300';
JResponse::setHeader('Cache-Control', $cacheControl );

// if content not public (browser_cachable==1) set an expires header to the past 
// to make sure that proxies will not cache it
if ( $browser_cachable !== null && $browser_cachable != 1)
  JResponse::setHeader('Expires', 'Fri, 01 Jan 1990 00:00:00 GMT');
// else set some expires header to the future, or do not set at all

I will welcome any change to Joomla to provide an easier way to do this. Someone with better knowledge can say

This topic seems to be about general Joomla Development, except for the part that Joomla always sets the no-cache, must-revalidate header and creates session cookie for unlogged users

just we do need a default header, that will work in ALL cases, and it seems that default header does (maybe a configuration option to disable default header)

There is no reason whatsoever to add something to the HTTP header. As I said before, I am doing it and it works perfectly fine. Additionally in SobiPro we are sending the currently visited section within the header as well. The header is there exactly for that reason ;)

about session cookie for unlogged users, the unlogged users also have a session so i am not sure if that can be avoided (can it ?)

IMHO we do not need to have the session always created. Moreover, it could also save some performance as well. As I understand it, we need the session only if some code storing something in the session. If the visitor not used the session at all, there is no reason to initialise it and store it.

So you cannot configure varnish server to decide by its own by looking if cookies exists,

Oh I could. But I don't think it would be wise. Instead Joomla! should be able to decide if we actually need it.

Regards,
Radek

Hello, the headers are set inside
joomla\application\web.php

inside frontend controller of the componet (display task), if you use:

JResponse::allowCache(true);

then the ELSE part of the following is called

protected function respond()
{
    // Send the content-type header.
    $this->setHeader('Content-Type', $this->mimeType . '; charset=' . $this->charSet);

    // If the response is set to uncachable, we need to set some appropriate headers so browsers don't cache the response.
    if (!$this->response->cachable)
    {
        // Expires in the past.
        $this->setHeader('Expires', 'Mon, 1 Jan 2001 00:00:00 GMT', true);

        // Always modified.
        die('reached web');
        $this->setHeader('Last-Modified', gmdate('D, d M Y H:i:s') . ' GMT', true);
        $this->setHeader('Cache-Control', 'no-store, no-cache, must-revalidate, post-check=0, pre-check=0', false);

        // HTTP 1.0
        $this->setHeader('Pragma', 'no-cache');
    }
    else
    {
        // Expires.
        $this->setHeader('Expires', gmdate('D, d M Y H:i:s', time() + 900) . ' GMT');

        // Last modified.
        if ($this->modifiedDate instanceof JDate)
        {
            $this->setHeader('Last-Modified', $this->modifiedDate->format('D, d M Y H:i:s'));
        }
    }

    $this->sendHeaders();

    echo $this->getBody();
}

About setting custom headers, i don't know where is the best place to do it, i do it inside onAfterRender event, you may make some pull request to make this simpler ?

About session, disabling session cookie for guests (by a global configuration parameter?), will make URLs get the a session variable, right ? (but not good for security)

• the above parameter should effect _setCookieParams() inside JSession
• also maybe it is acceptable "silently" not to have not session for guest, and any code that will set data into it will have these data "silently" disappear (i mean a site-maker will know what he/she is doing)

i do not know enough myself, again if you make a pull request maybe it will considered, you know more of these than me ))

Regards ))

@Radek-Suski take a look at the comments on the top of this snippet: https://snipt.net/fevangelou/the-perfect-varnish-configuration-for-joomla-websites/

@dgt41 I was trying it before. It doesn't work with Joomla! 3.x

I was finally able to configure it that way that visitors are getting cached version of the site but this is tailored for a particular website only. Not a general solution.

@ggppdk

About session, disabling session cookie for guests (by a global configuration parameter?), will make URLs get the a session variable, right ? (but not good for security)

I don't think so. But I may be wrong.

Would be nice if someone who knows it exactly could give some hints ;)

I also think Joomla should be easier to configure with proxy/cache servers like varnish, nginx, etc.

Caching with proxy/cache servers gets even even worst in multilanguage sites, because the language plugin sends a cookie to the client with the user language, and, that i know of, there is no simple way to remove that cookie.

There is also the problem of votes, visits count, etc in modules and articles. But that you can only resolve with ajax calls or by cache warm up techniques.

I don't know very well varnish, but nginx cache store the HTTP Header + HTTP content in a file for caching, so if there is a HTTP Header "Set-Cookie" with the session it will store it and send that session cookie to all the clients.

In a project i had to do a very costumized and complex setup for making it work with nginx cache (nginx config and a custom Joomla system plugin). I used a mix of Joomla sending custom HTTP header like "X-Accel-Expires" to send the cache expiration time and custom cookies like "X-No-Cache" and removing the session cookie when caching the page in nginx.

I did that because was needed for the guest user to always have a cached page except when is login in, is logged or is logging out. Also i didn't want the contact page and some custom pages to be cached.

I think that if it's for caching nginx doesn't need the session cookie because nginx will send simple HTML to the clients. Nginx doesn't even connect to the PHP-FPM server when sending a cached page. So no PHP session needed.

If i'm not wrong, Joomla still needs some improvement to make it work in a simple (or at least not much complex) way with proxy/cache servers. And some documentation too.

@Radek-Suski

If it's any use, I've done something very similar with NGinx - https://www.bentasker.co.uk/documentation/joomla/219-making-your-joomla-site-fly-with-nginx-reverse-proxy-caching - though my config has had a few tweaks since I published that.

The main basis of controlling the cacheability is a OnAfterRender plugin, I told the cache to disregard the origin's cache-control headers (though they're still passed downstream).

There is also the problem of votes, visits count, etc in modules and articles.

I took a fairly ungraceful way of handling visit counts - access logs are parsed hourly (as I'd be running them hourly anyway) and for every cache hit, a cache bypassing request is made to increment the hit counter.

I don't know very well varnish, but nginx cache store the HTTP Header + HTTP content in a file for caching, so if there is a HTTP Header "Set-Cookie" with the session it will store it and send that session cookie to all the clients.

You can conditionally strip the set-cookie header if you consider the page cacheable. It's not ideal, but it does the trick.

If i'm not wrong, Joomla still needs some improvement to make it work in a simple (or at least not much complex) way with proxy/cache servers. And some documentation too.

Agreed, it's possible to get Joomla working with reverse proxies, but it's not as straight forward as it should be, and you're largely on your own for working various things out.

Hi you created this issue sometime ago but have not provided any code for people to evaluate. As no one else has shown any interest in providing the code and you have not then I am closing this issue at this time. If code is provided (a pull request) it can always be re-examined.

Hey,

Another vote for better header handling options. If you set up a "static" website with no dynamic content and no user login, there's no reason NOT to use public caching.

I'm trying to set up a joomla 3.7 website behind cloudflare, and use the "cache all" option on cloudflare, but it does not cache because Joomla sends those no-cache headers.

Is there any way to remove those headers ?

Thanks
L.

@plancked can you please open a new Issue as there is low Notice if writing on closed Issue, thanks.