Tips to handle Request / Response size more than 8K in Akamai-Apache-Adobe(AEM) Set up

In general web servers implementing http 1.1 protocol does not place any prior limit to the length of characters in URI (means GET request) and post requests. So practically all the modern browsers built on HTTP1.1 should not restrict any length of data, but servers implementing this protocol restricts the data size if URI size more than 255 bytes.

In all practical purposes this solution is tested in Akamai (as CDN), Apache as (Proxy/Reverse Proxy/Web server) and Adobe-AEM (as CMS). But the solutions are generic in nature and can be applied to individual set ups or combined as whole. 

Servers like Apache Web server, Akamai, Adobe AEM are only allowing 8KB of data in request and response. If the data size is more than allowed length we should get an error "400- Bad Request" or "414 - Request-URI Too Long". 

If you are getting 400- Bad Request because of sever has set limit either in proxy/reverse proxy whereas if we get 414 - Request-URI Too Long means the URI size need be increased. 

Here are some server specific use cases/solutions to   400- Bad Request and 414 -Request-URI Too Long.

Use case 1- 400-Bad Request - Adobe (AEM)

If request/response which involves a large amount of data transfer through AEM publisher then increase the default size of 8kb data otherwise you will get 400-Bad Request.

The fix depends on versions of AEM/CQ

If The version is 5.6 or lower than go for the following steps

•      Go to https://www.xxxx.com/system/console/configMgr
•      Search CQSE then and click on the link Day CQSE HTTP Service
•     A new pop-up window will be opened. Change the value of "Request Buffer Size" to the size you need. default is  8KB
•      Click save button. AEM/CQ can now handle the large amount of data what you set in AEM/CQ. Refer to the picture AEM/CQ 5.X

                                                                 AEM/CQ 5.X 

If The version is 6.x or higher than follow the following steps

•         Go to https://www.xxxx.com/system/console/configMgr
•      Search “Apache Felix Jetty Based Http Service” then and click on the link “Apache Felix Jetty Based Http Service”.
•      In 6.2 the solution moved to Felix jetty and jetty provides two fields “Header Buffer Size” and “Request Buffer Size”.  The “Header Buffer Size” provides 16KB by default and “Request Buffer Size” provides 8KB. So, if total request size is more than 24 KB then increase ““Request Buffer Size”.
•         Click save button. AEM/CQ can now handle large amount of data what you set in AEM/CQ.  Refer to the picture AEM/CQ 6.X

AEM/CQ 6.X

Use case 2 - 400- Bad Request - Apache Dispatcher-Adobe (CQ) integration

If request/response which involves a large amount of data in an Apache(Dispatcher)-AEM/CQ integration scenario, in that case, AEM/CQ dispatcher has a size limit all the URLs goes through it. Adobe provides a dispatcher that says it has the fix to handle more than 8 KB but did not work for us. To fix this

          2.1 - remove the URL pattern from dispatcher.any so that that request does not go thru dispatcher

 /filter

      {

      #/0104 { /type "allow" /glob "* /hreq/myapi/*" }

      /0041 { /type "allow" /glob "* *.css *"   }  # enable css

      /0042 { /type "allow" /glob "* *.gif *"   }  # enable gifs

      /0043 { /type "allow" /glob "* *.ico *"   }  # enable icos

     }

If you see #104 says any request with pattern hreq/myapi will not be allowed through            dispatcher.

2.2  -Write the proxy rules directly access publisher url instead of going through dispatcher

      RewriteCond %{REQUEST_URI} ^/hreq/myapi/(.*)$

       RewriteRule ^/hreq/myapi/(.*)$ http://xxxxxx:6503/hreq/myapi/$1 [P,L,QSA]

Here any incoming request to Apache will directly go the IP (xxxxx:6503) for any request comes for "hreq/myapi" and will not go through dispatcher. This can be mapped to domain URLS if have that set up in prod server like

  RewriteCond %{REQUEST_URI} ^/hreq/myapi/(.*)$

  RewriteRule ^/hreq/myapi/(.*)$ http://mydomain.com/hreq/myapi/$1 [P,L,QSA]

              We need 2 (2.1 and 2.2)steps to be fixed. Fixing one step will not resolve the issue

Use case 3 - 400- Bad Request – Akamai

If your domain is hosted in Akamai then your request/response will not be processed because Akamai has a limit of 8KB header size. If you get error 400 Bad in the browser and in Akamai get the reason is Request "R_FWD_BAD_HEADERSInvalidForwardReply: The forward reply headers were malformed or too large”. The error will occur when the HTTP response status line is missing or malformed or when the forward response headers exceed network:http.response-headers.max-size which defaults to 8192 bytes."  then it is clear need to increase header size in akamai. So increase Akamai header size .

Use case 4 -414 - Request-URI Too Long - Apache

If you URI contains large amount of data then Apache will return 414-Request-URI Too Long.The fix is in Apache config file(httpd.conf) increase the size by adding following line

LimitRequestLine 16384

This will increase the size to 16KB. If you set 

LimitRequestLine 0 then the size will be 2GB and that will be maximum allowed size. Default size is 8Kb.