Monday 30 July 2012

Certificate errors with Safari, App Store, iTunes etc

Being in the tech support business I like to follow how technical issues are examined and resolved. With the advent of support forums this is often done in public where kind hearted techies try to assist general folks while the Ingnorarity whine "Vendor x should just fix this." or mutter vague unsubstantiated threats of "I will never buy Y again because of this issue."

The best type of issue to observe is the type with
  • Random and conflicting symptoms,
  • Tied to a "feature" introduced in some basic piece of infrastructure that everyone either ignores or just does not know about.
  • The failures can be bypassed but cause ongoing annoyance and concern.
Apple OSX is currently struggling with one such issue handling web site security certificates.  I have seen this one myself on an up to date MacBook Pro laptop.  Websites that need to be trusted (banking, paypal etc) barf certificate error messages and the AppStore gives "Connection error" when trying to update software even though iTunes store logs in correctly with the same AppleID. 

Some background information on Website certificates

Website certificates are part of the trust mechanism that tries to ensure that the website your talking to is the actual website and when talking to that secure website, no one snooping can see what your talking about.  Without going too deep on this, trust certificates are built into programs and websites which are then used to set up and validate secure end to end connections. Most peoples experience of these infrastructure operations extends to seeing the "S" in a secure web address such as httpS://www.xyz.com.   See this for yourself go to www.ebay.com and click login. The webpage address at the top of the page changes to https://signin.ebay.com/... In the background an encrypted secure SSL connection is set up and used to pass the login details and password to the website using encryption.  Anyone listening into your data conversations would see the connection to the websites being made but are not be able to read the user names and password as they are passed to the website.

The SSL connection and website certificates mechanisms are based on a public/private key crypto system. This system works on the basis that if you encrypt information using a publicly available key, that information cannot then be decoded unless you have the private key. The public keys are distributed freely to programs that need to send information but the private "Decode" keys are held safe at the receiving organisation.  Anyone intercepting the message may know the public key and the encoding mechanism but will not be able to read the message without the companies private key.  Think of a special padlock with two keys. One key can only lock the padlock and the other can unlock.  The padlock and the locking key is freely distributed but the unlock key is held securely.  Using this special padlock, goods can be secured in transit.  The trust certificates ensure that your are getting the public key from the correct organisation and not another pretending to be the receiving organisation.

Part of the trust certificate system is a mechanism to revoke and cancel certificates that have been stolen or broken.  Amazingly this revocation system was not implemented in some operating systems certificate handling code until recently.  Full use of both certificates and the revocation mechanism has long been a requirement of even mildly secure internet security protocols.  Recent break-ins to organisations that really should know better has reenforced the need for both secure certificates and a revocation mechanism.  Read more about CRLs and OCSP to get a view on how some of this technology works.

Apple's certificate problem

When browsing and trying to login to sites that use the above httpS mechanism would sometimes get an error message similar to "This certificate was signed by an untrusted user." Looking at the Safari activity window a similar message is seen.  In this example the site certificate https://www.paypalobjects.com/ that was issued by Verisign does not work correctly. The web page behind showed with the correct details from Paypal but the graphics from  www.paypalobjects.com missing.









This problem also shows up when using the Appstore to get software updates.  After selecting an update, an Apple_ID login box appears. Logging in using the same credentials that work correctly for iTunes gives the cryptic error message "Connection failed."  The problem seems to have first shown up in the recent OSX  10.7.4 update.

This is a type insidious error that causes a loss of trust in both the platform being used and the apparent "security" of the internet for Mac users. Having to bypass bogus certificate errors on a regular basis trains users to ignore the unusual and vastly increases the chances of accepting a hostile or correctly revoked certificate.

Looking at the Apple forum entries on this issue, amongst the normal forum noise and confusion, the following suggestions can be seen:

This is a system clock issue: True, if set a long way out, can cause certificate to appear as expired when they are not. This does not cover the "invalid issuer" messages that are at the heart of the issue.

This is an ocspd issue:  Kind of in the right area, it is a certificates problem. However the suggestion that followed to change the preferences in Keychain application to switch off the certificate revocation mechanisms CRL and OCSP. This is madness do not do this.  This is the sort of Doh! advice that goes alongside "Never forget where your car keys are by leaving them in the car."











The normal setting for the above items are "Best Attempt".

What fixed it for me

Looking down the posts the following entries by Apple forum user quickSti
fixed it for me and others. It did take about 10 minutes poking into the unfamilier Keychain application but has fixed the web site certificate ( invalid issuer )  error and I can now login to the AppStore.

Good Job quickSti.

quickSti
I solved this on my wife's computer by resetting the security certificate settings.  This might help others:
Close all (Safari) windows.

Application -> Utilities -> Keychain Access ->  click on System Roots on the left, and then click on Certifcates on the bottom left.

Check to see if any of the certificates on the right have the blue "+" symbol - this means they have custom trust settings.
There is a bug in changing the policies, so you'll have to change them via the method below. Changing them just by changing the access to "system defaults" doesn't seem to save.  The method below worked for me.


Double-click on each certificate with the custom setting (blue "+"), expand the section labled "trust".  Change the "Secure Sockets Layer (SSL)" setting to "no value specified".  Close window - you should be prompted for the password.  Double-click on the certificate again, expand trust, change the "When using this certificate" setting to "Use System Defaults".  Close window, and re-enter password.

If you didn't re-enter your password upon closing the window, the setting didn't take.  The blue "+" should disappear after a few seconds when it's set back to default.  Once all of the certificates are changed back to default, restart Safari.

This solved all of the problems for my wife's computer with these issues and OSX 10.7.4

This Explanation expanded on the solution

This worked for me.  It's ingenious since there was some kind of bug with the 10.7.4 update and it caused certain certificates to change their trust values or not register them properly in Keychain Access.  This method below has you change a certificate's trust value, save it, then change all the certificate's trust values back to default as it should be.  This solved my problem and my father can use his banking websites on Safari again.  However, it does take some time to enter and re-enter the admin password again and again for over 20 certificates.  Good luck and thank you for the advice.




There you have it, an ugly infrastructure bug that impacts normal web operations in an annoying and trust reducing way. A few red herrings seen on the journey to resolution. For sure this is not full root cause but enough for most folks to get by.  There are other similar looking error messages so please take care to ensure that what your seeing is the same if you expect the same solution to work for you.  


Finally remember to capture a screen shot using Grab or similar. When working with technical support folks, showing what you see is far more compelling than a vague description of a half remembered message.


Cheers


Gannett