Vanity A/B testing with page caching

Question

I'm using Vanity to do my A/B testing but I also have page caching for the pages the experiment is running on. Because of that, the user will actually see both options as they navigate around the site. I would be fine with that however I know that Vanity keeps track of which option a single user is seeing and attempts to show that same option throughout the site.

So what I'm wondering is, let's say a user comes to my site and views Option A. Vanity (I think) records them as an "Option A" participant. Now that user goes to a different cached page, and because of the caching they see Option B and "converts".

Is vanity going to record that user as an "Option A" conversion or an "Option B" conversion?

Answer 1

我与Vanity的创建者进行了交谈，为获得准确的结果，关闭要进行A / B测试的页面的缓存确实很重要。

Answer 2

I've recently started helping to maintain Vanity.

This question is similar to the issue reported here , which was recently addressed and closed by PR#144 (just merged onto master, will be included in the next release).

A cache string for a user can be generated using (from a controller):

Vanity.playground.participant_info(vanity_identity)