Introduction

[0:00:14] ADRIAN GARCIA RAMOS: So, welcome to Pathmonk Presents, where we talk with growth and marketing leaders about what's actually working right now. I'm Adrian, your host, and today I'm joined by Deborah OMalley, Founder of Guess the Test. Welcome, Deborah. Thank you. How are you?

[0:00:32] DEBORAH OMALLEY: PJ.

[0:00:34] ADRIAN GARCIA RAMOS: Great, great, it's good to have you here. So… what is… Guess what that's about?

Guess the Test and the Pipedrive checkmark test

[0:00:44] DEBORAH OMALLEY: So Guess the Test is an experimentation resource. Can I share my screen?

[0:00:49] ADRIAN GARCIA RAMOS: Oh yeah, absolutely.

[0:00:50] DEBORAH OMALLEY: I'll pull it up for you, and I'll share the screen. This may be… we can edit this part out, but, is it mostly audio listeners, or is it mostly video?

[0:01:01] ADRIAN GARCIA RAMOS: It's… it's going to be both of them, so I would recommend you to give a kind of explanation so anyone can understand what you do, and then, you know, we can also edit on top of the thing and show video, so… Feel free.

[0:01:17] DEBORAH OMALLEY: Okay, great. Alright, so we'll start again from the top here. Guess the Test is an experimentation resource. It's dedicated to helping you make money through A-B test case studies. By inspiring your A-B test ideas and validating your A-B test ideas. So, on the site, there's tests from big-name brands, like Tommy Hilfiger, Culligan Water, T-Mobile, and every month there's a new, different, featured test.

[0:01:49] ADRIAN GARCIA RAMOS: F?

[0:01:49] DEBORAH OMALLEY: behind the curtain, so we'll pull back the curtain, and why don't we take the featured test together? Would you like to be put on the spot, Adrienne, and see if you can guess the right test?

[0:01:58] ADRIAN GARCIA RAMOS: Absolutely, why not?

[0:02:00] DEBORAH OMALLEY: So… more, and see if, Adrian, you can guess the right test. So, in this test, the team wanted to know… I'm actually going to go into incognito mode so you can take the test and… Guess the right answer.

[0:02:14] ADRIAN GARCIA RAMOS: Right.

[0:02:14] DEBORAH OMALLEY: wanted to know. Is it better to use information tooltip icons.

[0:02:23] ADRIAN GARCIA RAMOS: can do.

[0:02:23] DEBORAH OMALLEY: Racing chart, like what you see here, these little eye bubbles. Or, is it better to use check marks, like what you see here? So that was really the only difference between the pricing chart, info bubbles, and tooltip icons. Right. The team was trying to increase… Leads? People who had already started a free trial of the platform.

[0:02:48] ADRIAN GARCIA RAMOS: Okay.

[0:02:49] DEBORAH OMALLEY: a platform called Pipedrive, which is a kind of, like, CRM and sales system.

[0:02:54] ADRIAN GARCIA RAMOS: Right.

[0:02:54] DEBORAH OMALLEY: People had started the free trial, and now it was time to them… for them to upgrade from free to paid. They had been through some of the funnel and some of the flow, they were familiar with the process, but now they were being prompted to make that decision if they were actually going to buy or upgrade the platform. So the KPI was free-to-trade paid trial conversions, and the overall goal was to determine which was better, the bubble.

[0:03:21] ADRIAN GARCIA RAMOS: This is where the.

[0:03:22] DEBORAH OMALLEY: techniques. Right. So, what do you think? Bubbles and check marks? Version A or version B?

[0:03:27] ADRIAN GARCIA RAMOS: I would say checks, but I'm not sure about that.

[0:03:32] DEBORAH OMALLEY: Okay, so you're guessing check marks, what's your…

[0:03:35] ADRIAN GARCIA RAMOS: Yeah.

[0:03:36] DEBORAH OMALLEY: Why do you think checkmarks?

[0:03:37] ADRIAN GARCIA RAMOS: Because they… at least me, I see them as something positive, and that could, in a way. you know, change my behavior in a way, yeah, I would say so.

[0:03:50] DEBORAH OMALLEY: Alright, well let's do the big reveal. See if you're right, and we're gonna see if you guessed the right test. Congratulations!

[0:03:59] ADRIAN GARCIA RAMOS: Okay.

[0:04:00] DEBORAH OMALLEY: You guessed So that is indeed what seemed to be happening.

[0:04:04] ADRIAN GARCIA RAMOS: Right.

[0:04:05] DEBORAH OMALLEY: Simply changing from info icons to checkmarks markedly increased conversions by 9.1%.

[0:04:15] ADRIAN GARCIA RAMOS: Okay.

[0:04:16] DEBORAH OMALLEY: Free-to-pay trial conversions, so it not only worked, it worked very well for the team.

[0:04:21] ADRIAN GARCIA RAMOS: Right. And we can't…

[0:04:23] DEBORAH OMALLEY: always know the why through quantitative analysis and A-B testing, but what this suggests is that providing that reassurance, as you said, those check marks, which is a very positive sign, they were.

[0:04:34] ADRIAN GARCIA RAMOS: Right.

[0:04:35] DEBORAH OMALLEY: So signals kind of go.

[0:04:37] ADRIAN GARCIA RAMOS: I'm listening.

[0:04:37] DEBORAH OMALLEY: to indicate to users that this was something that they should proceed with. The icons probably caused a little bit of stalling, and like, hey, wait a minute, should I be thinking about this more? Should I be learning about this more? And it probably caused a little bit of confusion and increased cognitive load at a key decision point when users had sort of already made the decision.

[0:05:00] ADRIAN GARCIA RAMOS: So… Right.

[0:05:01] DEBORAH OMALLEY: This shows us that at high intent places in the conversion funnel, we shouldn't automatically assume that more information is better, especially… for users that are already primed, anything that we can do to reduce pain points, remove distractions, and make the purchase path smoother and easier is absolutely what we want to be doing. And even just a tiny, small UX element, like a green checkmark. accomplish that, and influence.

[0:05:29] ADRIAN GARCIA RAMOS: Okay, okay, so I've got a couple of, I mean, questions, especially looking at what you're just sharing, right? Sure. I'm wondering here, like, do you try to, in a way, be a training or kind of a sharing insights with people visiting your website? Or is this kind of a case of a study that you just wanted to show us in terms of how the platform works?

[0:05:56] DEBORAH OMALLEY: Yeah, this is a case study that was submitted by the testing team, actually, in this case at Pipedrive. Right. There's hundreds of case studies like this that are.

[0:06:06] ADRIAN GARCIA RAMOS: Okay.

[0:06:07] DEBORAH OMALLEY: by organizations and experimentation teams from all.

[0:06:10] ADRIAN GARCIA RAMOS: Right.

[0:06:11] DEBORAH OMALLEY: Every industry vertical, every topic that you could possibly.

[0:06:14] ADRIAN GARCIA RAMOS: Absolutely amazing.

[0:06:15] DEBORAH OMALLEY: And…

[0:06:16] ADRIAN GARCIA RAMOS: That's right.

[0:06:17] DEBORAH OMALLEY: What you can do is you can look for tests and case study ideas that interest you, you can sort test patterns if you're interested in a particular idea, or maybe a conversion goal, or industry vertical, and sort for you to find things that are relevant to your industry and the kind of work you're doing.

[0:06:35] ADRIAN GARCIA RAMOS: Right.

[0:06:35] DEBORAH OMALLEY: Ideas and an inspiration.

[0:06:37] ADRIAN GARCIA RAMOS: Sometimes.

[0:06:38] DEBORAH OMALLEY: I find it can be really useful if you're working with a client or a stakeholder who's resistant to an idea that you might have. You can pull up Guess the Test and say, well, look, another organization already did this, and they were able to either win or not win with it, and so this may be a good idea for us to test.

How each case study gets a trustworthiness score

[0:06:56] ADRIAN GARCIA RAMOS: Absolutely. I've got a question here, and that relates to the data, right, that you're using, because at the end of the day. if I come to your website as a user or a potential customer for the whole thing. I would wonder, like, is it public, the data that you are, in a way, sharing? So, am I going to know, like, who's that belonging to, and… When was that, in a way, you know, tested?

[0:07:25] DEBORAH OMALLEY: Definitely, yeah, really good question, important question, and so we'll just go back to the test that we had been looking at now.

[0:07:32] ADRIAN GARCIA RAMOS: Right.

[0:07:33] DEBORAH OMALLEY: Every study that is published on the site has a test trustworthiness kind of analysis, and.

[0:07:42] ADRIAN GARCIA RAMOS: Okay. Yep.

[0:07:43] DEBORAH OMALLEY: It's pretty long, so you have to open the accordion, because it takes up quite a bit of space, but it evaluates how trustworthy the test was run, based on, number one, the amount of data that's reported by the client, and some.

[0:07:57] ADRIAN GARCIA RAMOS: Cleans.

[0:07:57] DEBORAH OMALLEY: Just don't report any data, so you have to.

[0:08:00] ADRIAN GARCIA RAMOS: Right.

[0:08:00] DEBORAH OMALLEY: what they're saying is valid. Some clients give all the data plus a results chart so you can verify.

[0:08:08] ADRIAN GARCIA RAMOS: price.

[0:08:08] DEBORAH OMALLEY: Let's see, I don't remember exactly how much data was given, but it was evaluated based on the sample size, the.

[0:08:15] ADRIAN GARCIA RAMOS: empty.

[0:08:16] DEBORAH OMALLEY: the minimum detectable effect, power, sample ratio mismatch, and statistical significance. So this study ran for 3 weeks, which is good. Ideally, you want to see a test between a 2-6 week time frame. According to what I've heard from the stats gurus out there, a minimum detectable effect of 2-5% is considered.

[0:08:37] ADRIAN GARCIA RAMOS: surprising.

[0:08:38] DEBORAH OMALLEY: So, everything is evaluated on the upper bound of 5%. In this case, the effector, the conversion lift, was about double that. It was 9%. So this suggests the study might have been a little bit underpowered. There might not have been a large enough sample size.

[0:08:54] ADRIAN GARCIA RAMOS: Right.

[0:08:54] DEBORAH OMALLEY: the study properly, and then it's evaluated on whether or not SRM occurred. In this case, the testing team didn't provide enough data to evaluate SRM. And then the results are independently evaluated through a statistical significance calculator. Again, this test, unfortunately, not enough data to evaluate. So an overall test trustworthiness score is produced. In this case, the results are somewhat suspect, but we could certainly look at other tests where full results are provided, and then it's much easier to evaluate whether or not the results were trustworthy. This test here has an example. We'll just go and see the results right away, rather than take the test.

They… we achieved a 2.79% uplift. They gave all the data here, they actually shared a screenshot of the results, and then… so it was way easier to evaluate the test trustworthiness.

[0:09:56] ADRIAN GARCIA RAMOS: Right.

[0:09:57] DEBORAH OMALLEY: that sample size calculations were met, SRM didn't.

[0:10:00] ADRIAN GARCIA RAMOS: Got it.

[0:10:00] DEBORAH OMALLEY: Her test statistical significance was achieved, and that kind of thing.

Where the tests come from: the Best in Test Awards

[0:10:06] ADRIAN GARCIA RAMOS: I wanted to ask you, Deborah, because you mentioned they. When you say they, it could be, like, anyone? Let's say you're looking forward to have the data within the library? Or is it rather you as a, let's say, a consultant, or your company itself who's, in a way, gathering all this data and formatting it and presenting it in a proper way?

[0:10:30] DEBORAH OMALLEY: Yeah, great question. So… I gather the case studies, people submit them independently. Generally, the bulk of the case studies are submitted through the Best in Test Awards, which is a yearly Or it may even become a biannual awards and event ceremony, where people submit tests, and then they're awarded prizes for the best tests and the most statistically rigorous tests. And then, throughout the year, they're published and shared on the site as a resource. So, for the most part, experimentation teams, or sometimes individual practitioners, are running the tests, and then they submit them to me. I take the data, and they answer a questionnaire as well, so I have a little bit of information about the study in the background. Sometimes I'll speak with them to get more information.

and find out how the test was run, and then I take that information and turn it into a case study write-up that you see here.

[0:11:25] ADRIAN GARCIA RAMOS: Yeah, absolutely. This is a, you know, an amazing work. I'm just stunned about it, to know about it. So, I wanted to know Since when you run these kind of, practice, in a way. And also, if you can let us know more about the award that you were mentioning before, because that could be very interesting for people, you know, running tests and… and applying for recognition and awards.

[0:11:55] DEBORAH OMALLEY: Definitely, yeah. So this site has been live since 2018, and so if you go through the chronology of tests, which are currently listed in chronological order, you can see that there's hundreds of tests on the site.

[0:12:10] ADRIAN GARCIA RAMOS: Right.

[0:12:11] DEBORAH OMALLEY: Drill down to the bottom here, and at the very bottom, as we keep going, going, going, you can select how many hits you want per page, and then we can go to the very last page here.

[0:12:23] ADRIAN GARCIA RAMOS: Right.

[0:12:24] DEBORAH OMALLEY: And you can see some of the very early first tests that appeared on the site. So since 2018, in terms of the Best in Test Awards, they… as I mentioned, happen annually, but we're looking at doing them twice a year now. This year, I partnered with Convert.com, which is a testing platform.

[0:12:45] ADRIAN GARCIA RAMOS: Right.

[0:12:46] DEBORAH OMALLEY: as well as Ecologics, which actually helps work on my site and helps make the site Functional?

[0:12:53] ADRIAN GARCIA RAMOS: Mmm.

[0:12:54] DEBORAH OMALLEY: And we hosted an awards ceremony where we had two categories this year. We had Best Educational Content, so this was everyone who's producing really strong educational content to propel the industry forward. best A-B test case studies.

[0:13:10] ADRIAN GARCIA RAMOS: Right.

[0:13:11] DEBORAH OMALLEY: Judges that are in the industry producing content evaluate the educational content.

[0:13:16] ADRIAN GARCIA RAMOS: And I looked.

[0:13:17] DEBORAH OMALLEY: at using astringent criteria for A-B test case study, and test trustworthiness, I looked at all the case studies that came in.

[0:13:25] ADRIAN GARCIA RAMOS: Right, right.

[0:13:26] DEBORAH OMALLEY: what was evaluated for the A-B test case studies. And then we had different announcers announce the award winners, and if anyone is interested in seeing the replay of the ceremony, it's available here.

[0:13:39] ADRIAN GARCIA RAMOS: Okay.

[0:13:40] DEBORAH OMALLEY: Link in the chat, and then you can also share it with your audience.

[0:13:44] ADRIAN GARCIA RAMOS: Yeah.

[0:13:45] DEBORAH OMALLEY: Yeah, anyone can watch the awards ceremony, get some really inspiring ideas for amazing tests that people are doing from a really progressive industry standpoint, and also seeing some of the great educational resources that are out there to help experimenters.

[0:14:02] ADRIAN GARCIA RAMOS: Yeah. Okay, I'm wondering… hmm… Many things, about the, you know, the, the full solution.

GA4, PostHog and AI in analytics

[0:14:16] ADRIAN GARCIA RAMOS: One of those is, the platform that you use to track all the events, right? Because nowadays, there are many, in a way, analytics tools that you may use for tracking. I would say that the most known one is Boss Hot these days. But I don't know if you are, in a way, affiliated or non-affiliated with them, or if you, vouch for, you know, another one. How… what's your approach on that end?

[0:14:48] DEBORAH OMALLEY: So, I take what I do with Guess the Test, and I apply it directly to client work. implementing the findings, the learnings, and the data-driven approach that I preached on Guess the Test, and work directly with clients. And so, with any client I work with, the very first thing that I do to audit a website and to work to optimize it is dig into their analytics data. What I find is about… which is in line with industry average. About 88% of clients have Google Analytics, GA4. And so that's what I've just kind of become most versed in, because that's what most people are using, likely, because… Right. I did have the good fortune of having one client that was using Post Hog.

[0:15:33] ADRIAN GARCIA RAMOS: Okay.

[0:15:33] DEBORAH OMALLEY: I really enjoyed postdoc, because you can do very quick AI, kind of, summaries or searches, or.

[0:15:39] ADRIAN GARCIA RAMOS: Oh, yeah.

[0:15:40] DEBORAH OMALLEY: Oops. and get things on the fly, like a conversion funnel, that you would have to go into GA4 and actually manually create a report for, and figure out all the steps, and, you know, sort of tag it and track it and all that kind of stuff, just automatically was created in posthoc. The data was a little bit suspect. I wasn't totally sure I could trust it, but it still was produced very quickly and really helpful as a result. So, I try and be platform agnostic. I'm not gonna promote one platform or the other. truthfully, I have the most experience in GA4. That's where most of my, sort of, you know. my clients are, so that's where I mostly spend a lot of my time. But I have used other analytics platforms, and I'm really excited to see the development of the analytics world as we start to bring AI into all the.

[0:16:31] ADRIAN GARCIA RAMOS: Right.

[0:16:32] DEBORAH OMALLEY: of the tools?

[0:16:33] ADRIAN GARCIA RAMOS: Absolutely.

[0:16:33] DEBORAH OMALLEY: I think it's really going to change the way data analysis is happening, and it's going to make it so that you don't have to necessarily be a data scientist anymore.

[0:16:41] ADRIAN GARCIA RAMOS: businesses.

[0:16:42] DEBORAH OMALLEY: have to go into BigQuery, or even all be all that versed in something like SQL or SQL. You'll just be able to do prompts, and the data will be analyzed for you, and then you can do what I do best, which is turning that data into meaningful stories that help you understand your users, so that you can optimize the site based on that.

CRO trends: automation and AI agents as users

[0:17:01] ADRIAN GARCIA RAMOS: Yeah, so you mentioned before that, the site is running since 2018, so it's been already, like, 8 years. I wanted to ask you about the current trends that you're seeing in the industry that you weren't seeing, like, you know, 4, 5, 6 years ago, and what are the most relevant, like, trajectories for CRO in these days?

[0:17:27] DEBORAH OMALLEY: Yeah, so obviously not… A surprising answer, spoiler alert, but AI is absolutely what everybody's talking about, and is kind of hanging over us as this big cloud of how is it going to affect the industry and experimentation in general, and is there even going to be experimentation? Is there even going to be websites, is a question. And how do experimenters deal with this kind of thing? So, I have a couple reports on AI that might be of interest to you and your audience. One is from the testing platform Chameleon, and assessed what they're doing with their AI prompting tool, PBX, and kind of gave it an evaluative score. It's a very in-depth report, I think it's 22 pages or something like that. And the other thing that your audience may be interested in is I've been keeping an experimentation tools directory.

Keep track of all the tools that are out there for experimentation, and…

[0:18:23] ADRIAN GARCIA RAMOS: Right.

[0:18:24] DEBORAH OMALLEY: The next thing I'd like to do is kind of go through and evaluate every tool and assign a quantitative metric to say how useful it actually is, because the.

[0:18:31] ADRIAN GARCIA RAMOS: Right.

[0:18:31] DEBORAH OMALLEY: These tools say they can do a lot, but, you know, when you actually.

[0:18:34] ADRIAN GARCIA RAMOS: Please.

[0:18:34] DEBORAH OMALLEY: Sometimes they can be a bit of a letdown. Sometimes they're great. But, there's a whole bunch of tools out there, as you can see. I'm scrolling through now, and we're just at B, and now we're going into C. This goes alphabetically, and it goes on and on and on and on, and I'm trying to populate it all the time as new tools come out, which is an exhausting job, because…

[0:18:55] ADRIAN GARCIA RAMOS: Hmm.

[0:18:55] DEBORAH OMALLEY: are coming up all the time. But the main synthesis of this that I'm seeing in terms of trends is that everything is moving towards automation.

[0:19:06] ADRIAN GARCIA RAMOS: Okay.

[0:19:07] DEBORAH OMALLEY: In the future, I expect experimenters will still be doing experimentation work, but a lot of the things we're doing manually is just going to be automated for us. Okay. Hypothesis generation tool will, you know, analyze the site, say, I think these are the opportunities, it'll probably launch and run the test for us automatically, and experimenters are going to be needed for oversight and that kind of human understanding and human psychology. But a lot of the, kind of, grunt work that we're doing now, or the busy work that takes a lot of time, is going to be eliminated through automation. The other question is, who are users? Right now, predominantly users are people, but in the near future, I anticipate that users won't just be people, they'll also be agents as well, or.

[0:19:54] ADRIAN GARCIA RAMOS: Right.

[0:19:55] DEBORAH OMALLEY: developments end up coming out of the tools that we have now. And so, if agents are talking to each other and using websites in a much different structured data way than people. then what is the paradigm for an optimized website? Is it one that appeals to a human user? Is it one that appeals to, you know, an AI user? Is it a synthesis of both?

[0:20:18] ADRIAN GARCIA RAMOS: Yes.

[0:20:19] DEBORAH OMALLEY: in between that doesn't even exist yet. And so I think we need to be cognizant and aware of these types of things.

[0:20:25] ADRIAN GARCIA RAMOS: Right.

[0:20:26] DEBORAH OMALLEY: to the new AI realm and try and optimize websites. I do think experimentation is going to have a foothold in the new AI paradigm. It's just going to be much different and much more automated than what we're doing now.

[0:20:39] ADRIAN GARCIA RAMOS: Right.

[0:20:39] DEBORAH OMALLEY: I worry for myself because I'm a bit of a dinosaur.

[0:20:43] ADRIAN GARCIA RAMOS: Yeah.

[0:20:43] DEBORAH OMALLEY: this almost two decades. I have very established practices. I have a huge repository here, and in.

[0:20:50] ADRIAN GARCIA RAMOS: Right.

[0:20:51] DEBORAH OMALLEY: And I think a lot of that knowledge is just gonna be, boom, automated by an AI agent, and, you know.

[0:21:00] ADRIAN GARCIA RAMOS: Yeah.

[0:21:01] DEBORAH OMALLEY: theory of the future.

[0:21:02] ADRIAN GARCIA RAMOS: Actually, you know, one of the features that we are bringing up, in Pathmonk is related to the… what we call the MCP for websites, right? So, at the end of the day, there are going to be human searches, but there are going to be also Agentic searches. Actually, as a matter of fact, every day there are more Agentic searches. So, what we are trying to do is to resemble, the websites as, human… treat them. For… for the agents. And we do that with this MCP protocol that is going to be available for, let's say, the accounts that use Pathmonk, and the idea is that the agents are going to be able to communicate

Regardless of what you've done with your website, right? So, I think that it's, 100% coming up. And we all should prepare for it, because it's… it's real. And, you know, it's happening today. People are buying on agents. So, we need to catch up with our websites, right?

Why personalization still has no clear winner

[0:22:15] ADRIAN GARCIA RAMOS: Yeah. I wanted to ask you, what do you think about, you know, the platforms offering, in a way, An automatic, you know, conversion rate optimization. for websites. You know, those, tools that, in a way, change the website automatically, depending on the visitor, What is your take on those, and what do you think is… you know, best or worst compared to just doing, like, the artisan of crafting every single experiment and, you know, testing. Every single thing, one by one.

[0:22:57] DEBORAH OMALLEY: Yeah, yeah, so they're certainly there. If you look at this list here, you can see that a lot of the tools are focused on exactly that.

[0:23:05] ADRIAN GARCIA RAMOS: strange.

[0:23:05] DEBORAH OMALLEY: To optimize and personalize the experience in real time for an individual user at scale. and then identify your geolocation or whatever attributes that it can find about you, and present a new experience based on that. I think this ultimately is the way forward in marketing and in web optimization. Although people have been trying to do it for a number of years, and nobody's really been successful at it.

[0:23:32] ADRIAN GARCIA RAMOS: Yeah.

[0:23:32] DEBORAH OMALLEY: I remember going to an Adobe conference in about 2016, and they said, we are trying to create a delightful experience through personalization, and that was kind of a buzzword back then. And really, personalization has never really reached its full potential or full capacity.

[0:23:50] ADRIAN GARCIA RAMOS: Right.

[0:23:50] DEBORAH OMALLEY: Well, it's not there quite yet, however.

[0:23:52] ADRIAN GARCIA RAMOS: Yep.

[0:23:52] DEBORAH OMALLEY: I do anticipate, as AI starts to become more sophisticated. of pipe dreams that people had in the past will actually turn into reality. And you, Adrian, are going to see a much different user experience. product showcase, then I am gonna see. based on your previous buying behaviors, based on what an AI knows about you, based on previous stored chat conversations, because, you know, I don't know about you, but, like, tools like Claude and ChatGTP, they have become my copywriter, my counselor, my astrologer, my tarot card reader, all these different things. They know all these personal, intimate details about my life.

And there…

[0:24:31] ADRIAN GARCIA RAMOS: Yeah.

[0:24:31] DEBORAH OMALLEY: I think my data, hopefully? But, you know, who knows?

[0:24:37] ADRIAN GARCIA RAMOS: Yeah.

[0:24:38] DEBORAH OMALLEY: And it's going to take all this data that it's amalgamated on each individual user, and then use it, undoubtedly, to show us the best, you know, shopping things So, a blue shirt, for example, let's just take something very simple. You're looking for a blue shirt, I'm looking for a blue shirt. Within your AI platform, which is going to become, like, our browsers, I anticipate, you're not going to have to go out to an outside website or experience, it's all.

[0:25:05] ADRIAN GARCIA RAMOS: Yeah, that'd be.

[0:25:05] DEBORAH OMALLEY: Presented there. There's already ads, for example, in ChatGTP, and, you know, I imagine it's going to become this nefarious thing where the highest bidder is going to have the top search result, and it's gonna.

[0:25:16] ADRIAN GARCIA RAMOS: Yep.

[0:25:16] DEBORAH OMALLEY: more prominent search result, and other results that may be more relevant will actually be suppressed because the advertiser isn't paying as much money. And so things are going to start to become very skewed based on the advertising that is available to individual users.

[0:25:33] ADRIAN GARCIA RAMOS: I've got another question related to the… the reason behind, not having a clear winner in terms of personalization, right? Because you mentioned it before, like, that conferencing with Adobe, like, 10 years ago, and this being all in all, for a number of years, and still, there is not something that you say, okay, I install it, and, you know, my conversion rate comes from 0.5 to 1.5. And it's really working for pretty much everyone. Why don't we have a clear winner over here, in your opinion?

[0:26:11] DEBORAH OMALLEY: Yeah, I think it's very, very challenging to personalize properly. In this day and age, with the tools and technology we have. And in order to truly understand the user, you need an awful lot of data points on them, because it's much more complex than just some basic demographic or socioeconomic information. Dynamic Yield, which originally started as Dynamic Yield, then was taken over by McDonald's, then was taken over by MasterCard. They're probably one of the most progressive examples of personalization, but even their personalization is still pretty rudimentary compared to What potentially could be done if you really kind of open your mind to the possibilities.

[0:26:55] ADRIAN GARCIA RAMOS: Absolutely.

[0:26:56] DEBORAH OMALLEY: I think we're gonna see more and better personalization in the future. And… companies are trying to do, they've been trying to do it for a long time. There's a test on Guess the Test. run by SAP back in, I don't know, 2020, something like that?

[0:27:15] ADRIAN GARCIA RAMOS: It's quite a while.

[0:27:15] DEBORAH OMALLEY: ago now, and it used IP address and demographic information to target the user and show them a different landing page image based on where they were coming from and what their industry vertical was. That was progressive back then. Now we can do so much more than that, because we can target all the different demographic points and geolocation points that we have about a user, and start to create a more fine-tuned and precise user experience. And as we have the technology to dynamically change and reorient and shift things on a site. I think we're gonna start to see hyper-personalized user experiences in the future. In fact, my personal prediction is within about 5 years or so.

[0:28:03] ADRIAN GARCIA RAMOS: I don't…

[0:28:03] DEBORAH OMALLEY: Websites as we know them.

[0:28:05] ADRIAN GARCIA RAMOS: Oh, yeah.

[0:28:06] DEBORAH OMALLEY: I think we're gonna see hyper-personalized Data snippets that are.

[0:28:10] ADRIAN GARCIA RAMOS: Yep.

[0:28:11] DEBORAH OMALLEY: to us that, you know, they're close approximation of websites, but.

[0:28:15] ADRIAN GARCIA RAMOS: You're not websites.

[0:28:15] DEBORAH OMALLEY: as we know them today.

[0:28:17] ADRIAN GARCIA RAMOS: Fair enough.

[0:28:17] DEBORAH OMALLEY: Traditional having to go out of a browser to a separate experience is going to become an outdated model.

Convincing clients that CRO matters

[0:28:24] ADRIAN GARCIA RAMOS: Absolutely. I've got a question for you regarding, the customers, right? The ones who recognize that have a conversion rate optimization problem. and assume or understand that they can, in a way, improve what they have in their websites, right? Because sometimes… When you talk to many people, depending on their websites, they are so focused on bringing more traffic to the website, or, you know, changing the pricing, or… it's just about, you know, the wrong things, when, at the end of the day, much of the traffic is being wasted. At a website, or at a, again, like a pricing page, or as you mentioned before, right, the buttons, or any other simple thing.

So, in your opinion, How the customers realize the problem and the opportunity of CRO.

[0:29:22] DEBORAH OMALLEY: Yeah, so this is a really important problem that I encounter a lot, particularly working with agencies that aren't specialized in CRO. I'm currently working with one kind of agency or client right now. who's specialized in paid media and SEM, search engine marketing, and they're great at getting traffic to the site, but that doesn't matter if that traffic is not converting.

[0:29:46] ADRIAN GARCIA RAMOS: So they…

[0:29:46] DEBORAH OMALLEY: brought me in specifically to turn that traffic that they're bringing to the site into high-converting traffic. And what I do is I take a very data-driven approach.

[0:29:56] ADRIAN GARCIA RAMOS: to look at.

[0:29:57] DEBORAH OMALLEY: what's working on the site right now, audit what's not working, use…

[0:30:01] ADRIAN GARCIA RAMOS: Right.

[0:30:02] DEBORAH OMALLEY: data, use heat mapping data, you use voice of customer, or any other qualitative. Data that's available to try and understand the pain points and the friction points for the users, and then to suggest ways to overcome that. And really, it's very simple, but my absolute best advice for anyone is to take a data-driven approach, and also to triangulate the data, or take a mixed methods approach, so you're using qualitative and quantitative data together.

[0:30:30] ADRIAN GARCIA RAMOS: Right.

[0:30:30] DEBORAH OMALLEY: Is the quantitative data, although powerful.

[0:30:34] ADRIAN GARCIA RAMOS: Olymp.

[0:30:34] DEBORAH OMALLEY: It doesn't tell you the what or the where of the problem, and it doesn't tell you the why, and that's where the quantitative data comes in.

[0:30:41] ADRIAN GARCIA RAMOS: Yeah.

[0:30:41] DEBORAH OMALLEY: Why people are behaving a certain way, because they tell you themselves.

[0:30:45] ADRIAN GARCIA RAMOS: Sure.

[0:30:46] DEBORAH OMALLEY: Bridge all those three things together, you get multiple data points that you can truly understand users, and…

[0:30:52] ADRIAN GARCIA RAMOS: Okay, so what do we do then with the customers who… need their eyes to be opened, right? Need to be enlightened, because they themselves don't realize that the problem that they have is the website and the commercial reorganization. How's…

[0:31:08] DEBORAH OMALLEY: Customers being the site owners, or the people who are buying from us?

[0:31:11] ADRIAN GARCIA RAMOS: Many customers to which we talked with, and no matter how you explain how important is the conversion on the websites. they're still focused on bringing more traffic to their websites, and spending more money in ads, and things like that. Because I guess that, for you, you kind of feel it, right? That sometimes. they don't realize the problem, and even if you take that approach, it's not, you know, enough somehow, so…

[0:31:41] DEBORAH OMALLEY: I think tools like Guess the Test can come into play. You can show those prospective clients or customers, here's some of the potential ROI that you can.

[0:31:52] ADRIAN GARCIA RAMOS: Right.

[0:31:52] DEBORAH OMALLEY: If you're optimizing your site. And, you know, it's really one simple line that you tell them, which is, you can have all the traffic in the world that you want, but if that traffic is not converting.

[0:32:03] ADRIAN GARCIA RAMOS: Right.

[0:32:04] DEBORAH OMALLEY: Why pay for it, or why work to acquire that traffic if it's.

[0:32:08] ADRIAN GARCIA RAMOS: I'm convinced.

[0:32:08] DEBORAH OMALLEY: And here are some strategies that we can use to increase your conversions with that traffic. you can sight-guess the test. You can also use ROI calculators that show if we increase conversions, you know, X percent, you can expect to see Y percent more revenue.

[0:32:26] ADRIAN GARCIA RAMOS: Right.

[0:32:27] DEBORAH OMALLEY: Usually, at the end of the day, money talks, and you have to speak the customer's language in terms of what's really going to motivate them. Is it conversions? Is it more money? Is it something else?

[0:32:38] ADRIAN GARCIA RAMOS: Good.

[0:32:38] DEBORAH OMALLEY: Usually, if you can tie it back to money in some way and tell them, you're gonna see… we can't guarantee this, but you're gonna see an X percent increase in your bottom line. then they'll go, okay, let's try this, let's see how it goes. And if you can prove results and make test wins based on informed, data-driven decisions.

The industries where CRO moves the needle

[0:32:56] ADRIAN GARCIA RAMOS: Right.

[0:32:57] DEBORAH OMALLEY: To build a case for yourself.

[0:32:58] ADRIAN GARCIA RAMOS: 100%. So, if you were to say 3, business verticals in which, you know, conversion rate optimization might… might really Move the needle, what would they be?

[0:33:14] DEBORAH OMALLEY: Every single industry vertical out there.

[0:33:16] ADRIAN GARCIA RAMOS: We're not limited to that.

[0:33:18] DEBORAH OMALLEY: 3. But if I had to choose 3, I would say e-commerce, for sure.

[0:33:22] ADRIAN GARCIA RAMOS: Okay.

[0:33:22] DEBORAH OMALLEY: B2B or SaaS products as well.

[0:33:25] ADRIAN GARCIA RAMOS: Right.

[0:33:26] DEBORAH OMALLEY: And with that, we're looking at a longer sales cycle. It's not necessarily directly tied to demos, usually it's getting qualified leads, a longer sales cycle, and then getting people to actually become paid customers from there.

[0:33:40] ADRIAN GARCIA RAMOS: Huh.

[0:33:41] DEBORAH OMALLEY: And then, I mean, those are really the two main.

[0:33:44] ADRIAN GARCIA RAMOS: Right.

[0:33:45] DEBORAH OMALLEY: But you have, sort of, ed, or ed tech, or not-for-profit, or government, and…

[0:33:51] ADRIAN GARCIA RAMOS: Okay.

[0:33:51] DEBORAH OMALLEY: those industry verticals as well. They usually don't always have a clear product to sell, but they're trying to either get leads, get engagement, or reduce their operating costs in.

[0:34:02] ADRIAN GARCIA RAMOS: Right.

[0:34:03] DEBORAH OMALLEY: And CRO can help with all of those things.

[0:34:06] ADRIAN GARCIA RAMOS: Okay, amazing. So… Thank you very much, Deborah. I think that it's been a super insightful conversation. Look forward… I mean, looking forward to knowing more about your awards and, you know, your site and all the examples that you've got. I think that leads and the materials that you've put together are just amazing. So I, highly encourage all our audience to visit it.

Where to find Deborah

[0:34:32] ADRIAN GARCIA RAMOS: I wanted to, you know, like, if you want to say, anything else about, you know, the… the whole ecosystem that you have created, or if the people want to, reach you out, what's the best way? So… Yeah.

[0:34:52] DEBORAH OMALLEY: Well, I'm always available on LinkedIn. I'll share my LinkedIn profile in the chat, and I can pull it up as well on the screen here. Linkedin is a great way to get a hold of me. I'm not as active as I used to be, just because I'm so busy.

[0:35:06] ADRIAN GARCIA RAMOS: Is he with that?

[0:35:06] DEBORAH OMALLEY: of stuff, but this is my profile. You can see I have over 26,000 followers, and that's because I produce and share really compelling educational content.

[0:35:15] ADRIAN GARCIA RAMOS: Right.

[0:35:16] DEBORAH OMALLEY: You can go through some of my featured content, as well as just the regular posts, and you can see all the stuff that I'm producing and sharing with the world, because I'm passionate about experimentation and getting it out there for people. So… connect with me on LinkedIn, check out Guess the Test, and I'm always happy to answer questions as well, and help people who are just getting started, or who have burning questions they couldn't get answered, because I know I certainly was in that boat. I still am. Sometimes I have questions, I'm just like, how do you deal with this? How do you answer this?

And I'll go from person to person to person. And, Adrienne, do you know the answer that I always hear? Tell me. It depends. And so then I try and figure out, well, what does it depend on? And I spend a lot of time digging and under… unturning stones to figure out what the actual answer is, and I tend to leave no stone unturned until I finally have the answer. So, that's my approach. Maybe that's what kind of sets me apart for success, is that I just will not take no, or it depends for an answer. I always am very curious to find out, and I'd like to share that knowledge with other people as well.

[0:36:27] ADRIAN GARCIA RAMOS: Fantastic. Thank you so much, Deborah, for coming on, and let's keep in touch.

[0:36:33] DEBORAH OMALLEY: Okay, that sounds great. Well, thank you very much. Pleasure talking with you.