Are certs cached when importing into cacerts?

Question

I imported a certificate into my /jre/lib/security/cacerts file using keytool. I tested my java application after importing the certificate( to prevent "PKIX path building failed") and it worked. Then I deleted the cert from cacerts and I expected to get the "PKIX path building failed" again, but instead it still works. How is this possible? I've terminated the JVM of my application and deleted the cert from cacerts.