Where our eCommerce development pipeline uses AI and where it doesn’t. Real examples from real work at Human Element.
Watch this episode on YouTube.
Dane Dickerson:
Well, okay, so I’ve got myself and I’ve got Kevin, who manages our development team, and we’ve got Eric, who’s our lead for project management here at Human Element. And today we’re talking through what we’re actually doing with AI as a company and how this is impacting the industry we’re in, work we do, and what we are trying to keep in mind to keep our usage of that always effective and useful for clients. Getting into the topic maybe can start with Kevin and then Eric, maybe a little intro about you and just what is the biggest thing you or your teams are having to do with AI these days.
Kevin Gardner:
So yeah, my name is Kevin Gardner. I am as Dane mentioned, I am the development manager here at Human Element. Probably when it comes to AI, AI, I’d like to say AI has revolutionized, you know, that’s like we want to use big buzzwords, like revolutionized how we how we do work. But considering we’re at Human Element, we tend to still want to hit on the human side of development. We’ve pivoted, it’s been an interesting journey overall with AI.
At first, when we found out that there were AI tools that could automate a lot of things that we do, we did what a lot of companies or a lot of just individuals do is like, okay, well then what can I get to do to get it to actually, just completely do particular tasks for me in which, in some cases, that worked out okay, but we found more friction overall with, confusion more than anything with developers thinking that, well, I have to use AI first and then I can use my own brain, right? Because I need to spend my time training this AI, which is essentially like having a junior developer who just happens to have all of the college knowledge just straight out of college and they have the, they just don’t have the experience, right?
Dane Dickerson:
Mm-hmm.
Kevin Gardner:
We started off with that focus more than anything with development. And for some folks it worked okay. And you were all different people, right? So people tend to adopt technologies and AI has some ethical concerns with it as well with certain folks. So we’ve had different levels of adoption, but more than anything now, looking back at the time that we’ve been working with AI, which has been a little bit now, more than anything, we’ve kind of rested in, use it for what you know that it works well for. And always be open-minded to try new things. Don’t get really stuck in a rut feeling like that AI can’t do a certain thing or it can’t help you. Always keep your mind open to what it could potentially do, but also don’t let it slow you down in the process just by trying to train it.
Dane Dickerson:
Okay.
Kevin Gardner:
If you feel like it’s something that you could potentially spend a little bit of time doing, it might be worth it because then you could pass that knowledge on to someone else along the line and tell them what you’re doing. And one of the things that I wanted to in my spiel here with more than anything is our team is very collaborative as you would imagine the development team is and the sharing around and just the, just the buddy system that we have around.
Just any of the knowledge that we’ve learned so far with AI has been pretty tremendous. It’s been tremendously helpful. One of the things that we have, we have an AI Center of Excellence repository where we put all the steering files for all the different platforms that we support, all the different tasks that we support, so that people can come in and they can grab all of these steering files and already have them pre-ready so that we can make sure that we are learning from mistakes and that we are learning from things that other folks have learned. So I think that the biggest thing with AI is you can do a lot, but you have to kind of still have to, even if it’s very, very smart, you still have to treat it like as a junior developer and it’s a junior resource in general and give it guidance the same way you would be onboarding someone for a new job. So it’s been an interesting journey and I think that we’re going to continue pushing forward with it and using it for what it’s good for.
Dane Dickerson:
Mm-hmm.
Eric Leslie:
From the project management standpoint, the development teams, know, ahead of a lot of our teams here at Human Element, we started with development using Kio and other AI tools. But what we’ve been focused on from the project management side is how can we support our downstream teams here at Human Element. identifying where AI fits into our processes, how to best use it, where, you know, similar to the developers, where can I do it as a human better than AI can do it or where does it fit in best? But what I kind of view as the objective is ensuring that downstream we’re helping out. So, you if we can have a tool like Kiro develop and come up with the framework and the additional information to help the developers do their job, that makes that process make sense.
As a project manager, you know, one of the things that I’ve really found is a good use of AI tools in my role is as a way to, from a higher level, take very complex tasks and break them down. so an example is one of our clients were migrating to a new platform and were migrating multiple business units. So it’s very complex and going feature by feature and what tickets, where do the dependencies lie? So what blocks what?
What enables another task and all those types of things, because we’re looking at every aspect of moving a feature off to the new platform, identifying eight or nine different tickets that are referenced within that single, that initial ticket and how they block each other. And those are things that AI is able to do a lot better and a lot faster than I am, for instance. So that’s where I find that AI is a good use.
In project management and keeping tickets organized and also, even just simple things like I’m between multiple Jira platforms for instance. So how do I keep the two in sync? What tickets changed in the last day as we have a team of seven working on this migration right now? So keeping up on all the Jira changes and everything else is quite a task. Using AI say, okay, in the last 24 hours provide a list of all the ticket changes with the last status that it is in so that I can eliminate some of the noise of all the status changes throughout a day for a single task. So those areas that I find that the project managers are using AI right now. But the main focus is identifying our current processes, where AI can fit in, but more importantly, how can it help downstream and also just use as an effective tool as a project manager without making it work harder.
Dane Dickerson:
Yeah, yeah, something for my team, which client strategy team has our SEO or advertising or design or marketing. And this year I turned 30 and that will also mark 14 years of doing marketing stuff and starting out in SEO. And for as long as I’ve been doing this, there’s always been a pretty large amount of what we’d call intern work.
And they were these tasks that we knew would be valuable, but are barely worth the time to do them. It does a lot of creating spreadsheets of matching up some values. It’s a lot of like collecting information from three sources and putting them into one. And something I did a lot of was writing spun articles, which is mostly you think of the past, but you’d write an article for a client.
And then you go through and write in all of the synonyms for words and phrases inside the article so that you could push out unique variations of it. This was like a content farming strategy that worked really well until it didn’t. But the things that put these tasks in common, not just that they’re barely worth the time to do them, it is that they are not problems that require a lot of in-depth knowledge or expertise and are tremendously boring. And so part of what these tools have enabled for us is to actually get completion on the tremendously boring work so that our team is then getting to the actual strategic decisions that come out of that kind of data.
If, yeah, say we’re assessing a client’s website, they have a need for better coverage of their products on the blog. Well, one thing I would have told an intern to do in the past would be to go take notes on all of their products and get me some data to use for an article for them. Well, grabbing that kind of especially structured data using some AI tools and getting to a first draft, that is lot easier now. And so it’s easy to make this sell to clients and do some of this work at scale so that we get to the real editorial and strategic decisions sooner. So it’s, you know, I think something that has helped adoption in this area is that sometimes you can face these tools and treat them as though they are expert resources. And in a lot of ways, they are the opposite. Like they will make foolish mistakes and inferences and don’t have a good grasp of facts. If you can approach these tools and be very aware of that, and now I need to be the source of truth, all this can do is gather up statements for me and put them together. Well, then you do have something pretty powerful. Just doesn’t work like a textbook or like Google or any of those.
Kevin Gardner:
Mm-hmm.
Great.
I did want to also add one of the things that I can see that is kind of, I don’t know if I would use the word hamstringing, but it’s kind of slowing down our adoption here at Human Element when it comes to AI in general, is just how technical it is and how technically adept you need to be in order to understand it. Historically here at Human Element and likely in a lot of other development shops, the developers,
Dane Dickerson:
Mm-hmm.
Kevin Gardner:
They do the development resources, right? The developers have the development resources like the repositories and things like that. And when we have non-development resources, they don’t need any of that stuff, right? They don’t do their job. They don’t really need access to any of that stuff. And let’s be frank, it actually saves money to not have everyone have access to that kind of stuff because that’s just less seats that you need to be able to do that. However, when it comes to how the current paradigm is with AI in general, regardless of what platform that you use, it’s very technically oriented. So you need to learn a little bit more about it. And then, you know, one of the things that we’re noticing is that there’s a big push to allow AI agents to be able to read websites. they want like Cloudflare added the ability to convert your HTML to be markdown readable so that you’re able to…so that AI agents are able to more quickly read because they don’t read HTML very well. They don’t write HTML incredibly well. So in my experience, working with some of our amazing, though we were on amazing non-technical resources that we have here at Human Element, passing one of those resources, a markdown file, it’s not quite like passing them another language.
Dane Dickerson:
Yeah, more quickly parse it and get.
Kevin Gardner:
It’s readable, it’s like you have to explain, here’s not only can you not just open it and understand the layout, you need to like get a reader of some sort. So it’s like, it has a little bit of a learning curve to it. So I kind of feel like AI in its very essence is supposed to make, at least the way I feel, it’s supposed to make my job easier. It’s supposed to like Dane mentioned, it’s supposed to make the mundane tasks.
Dane Dickerson:
Mm-hmm.
Mm-hmm.
Kevin Gardner:
…less mundane or get rid of them so I don’t have to worry about them so that I can use my brain power to focus on things that actually matter. And whenever it’s like, okay, we’ve got this new tool, I got to teach you a little bit on how to use it. think that AI needs to, moving forward, some of the newer AI platforms are starting to tackle this, but becoming less technical and more user-friendly, like pulling in a little more of, I mean, to name a few, like Gemini, for example.
Gemini is one good one that does a good job of pulling in all of your Google resources in order to figure out what you’re talking about, the history of what you’ve talked about in the past, to just talk to you. And you can just talk to it, and you don’t have to know anything technical. It’ll create you a spreadsheet. It’ll do whatever you need it to do. Whereas if you even get anywhere near close to where you want it to really start writing code or anything like that, then when you’re trying to communicate with non-technical resources, you have that kind of a divide, like a void, where you kind of have to backfill with knowledge in that sense. So it’s an interesting one.
Dane Dickerson:
Mm-hmm. Yeah.
Yeah. An illustration I think of, it applies here, but it’s the idea of design done by engineers looks like an airplane cockpit, and design done by designers looks like your car.
Like you sit down in an airplane cockpit and you’re faced with dozens of dials and switches. There’s like a five step process to get anything going. And it’s that way because that’s what makes sense to the people who built the machine. And that there’s something similar to that with a lot of how to use, especially the more powerful AI tools like, like Kiro and these assisted coding tools is they are designed to be used by people who are really comfortable with command line interfaces, with programming, with software. And yeah, for a large majority of people, well, and more importantly than just our company, but the vast majority of customers of our clients, that’s not something they care about, nor do they need to. ⁓
Kevin Gardner:
Mm-hmm.
Dane Dickerson:
Now, if you have the software brain personality quirk, which most developers have, and I’ve got a little bit of it, you think of things that you do in your life in terms of how could I automate this? And maybe my life is just a series of connected databases, and if I can get all the information from one spot to another, all of my problems are solved. If you think that way, you’ve already been pretty good at writing code, and this makes you better at getting there faster.
But if you don’t live that life, your life that way, you’re normal. But these tools aren’t going to hit the same way. You need more output focused interfaces.
I think so. I don’t know, Eric, you’ve used computers for quite a while. How do you feel about command lines?
Eric Leslie:
I actually prefer the command line interface for Kiro versus the IDE interface. I bounce back and forth a lot.
Dane Dickerson:
Yeah, but you’re comparing IDE to command line. These are still too developer-focused.
Eric Leslie:
Yeah, they’re very technical exactly. And so what we’ve been doing, what Dane’s team has been doing is kind of creating some tools to make things simpler for people like our account managers and project managers and maybe even some of the strategists, but the non-technical people that we’ve been talking about and how do we interface better. So looking at ways to not only streamline the process and fit it into our other processes, but make it very wizard driven. So what are you trying to do? I’m trying to write up a ticket.
Kevin Gardner:
Yes.
Eric Leslie:
Okay, well, what client do you want to do it for? I want to do it for this client. What are you trying to do? And, okay, well, even going to the fact of, I see you have access to the repository. Let me take a look there. it’s a bug with this feature. Well, let me take a look so that I can give the downstream developer additional information on where to look. And then even thinking so much as, and okay, well, let’s, you know, I have access to Jira, so let me write up the ticket for you. Confirm that this is the project that it goes in. Yes, it is. What’s the priority? Set it as high.
⁓Is it approved or do we have to get an estimate approved by the client? Do you have someone you’re ready to assign it to? All those things and then, okay, I’ve got everything I need, let me go to work. But making it very easy and driving the process for those non-technical folks.
Kevin Gardner:
Yeah.
And one of the things that I’ve noticed is AI at the very least, mean, it is, it’s maturing, right? But it’s far from mature, far from what we’re going to be working with the three of us when we’re older, right? You know, I’m sure it’s going to be a lot more integrated. But, know, like with project management, for example, they need a more polished finished product as a deliverable so that they can work with it, right?
Dane Dickerson:
Yeah.
Kevin Gardner:
The very nature of AI as it is, is discovery. We’re discovering, we’re learning new things, we’re figuring out new ways of doing it. And our developers in general, I’m not really trying to set it up as like a division here or anything like that, but it’s our developers are more inclined by our very nature to be more experimental and more like in discovery. But when it crosses into the line of like needing to be a product for our delivery process.
Dane Dickerson:
Okay.
Kevin Gardner:
Which is what Eric lives in, right? ⁓In the delivery process, he needs, like he mentioned, he needs a wizard. He needs something that’s polished and good to go. And that the interesting comparison has been that, you we want to work on things like that, like Dana’s helping Eric work on wizards, but that could be obsolete in two weeks, right? Because of the way that AI works. So it’s almost like you have to adopt…And I’m not saying that, you know, delivery can’t do this. I’m just saying it’s almost like we have to encourage folks to adopt a more like, you know what was kind of by the seat of our pants, minute to minute. And it’s very hard for the business portion of a business to operate in that, in that mindset, because you need predictability. Clients are looking for predictability. They’re looking for reliability, predictability. They need things to be consistent. They needed things to be deliverable and AI helps with that in its own way. So it kind of goes back to using AI for what it’s best for and realizing that it might be great now, but in six months you might not want to use it for that anymore. Which is kind of through our business for a loop on many occasions in trying to adopt AI.
We’ve had a lot of frustration to say the least with trying to adopt workflows because we’ll find that they work and then they don’t work or it doesn’t make sense or now there’s something new and then we need to try this so for us for the company the size of human element it is a challenge to continually try to innovate but we do we do continue to push on that
Dane Dickerson:
Mm-hmm. Yeah. Something I think about is the two, like two of the pitfall loops that the AI tools at this point in time tend to fall into is either the problem in front of me is too hard and it will refuse to do it or fail in the same way over and over. Or the problem in front of me could be solved with more information, but instead of that I will… try to solve it over and over and over and run into a loop. When, especially in the second case, nine times out of 10, the best thing for me to do is to interrupt the loop and state, no, the staging URL is this, stop searching documents. ⁓ Or that’s not how you access this information. That’s not how the system works, but something’s been severely misunderstood.
Kevin Gardner:
Right?
Dane Dickerson:
And so like they are different problems to expect to happen in the process and getting used to what those problems are going to be as a lot of the learning curve. This makes very different mistakes than you will. It’s not a person. It’s very elaborate, autocomplete.
Kevin Gardner:
Yes.
Eric Leslie:
Well, that’s kind one of the things that I sometimes struggle with. it signifies the importance of checking the work. As an example, I use the example of showing me ticket status changes. Well, we include an alternate ticket number. And it was transposing numbers. like, no, I don’t think you’re doing this right. But I’m used to Excel. A formula is a formula is a formula. The result is always correct. And it’s like, well, if Excel gets it, that’s.
Dane Dickerson:
Yeah, it’s always wrong in the same way. Yeah.
Eric Leslie:
Right, exactly, Once you get the formula nailed down, you’re confident in it. it’s like, I wouldn’t even think that I’d have to check three numbers, you know, the numbers across. It’s very simple. It’s right here. Just bring that in. And some of the mistakes that I see, it’s like, man, this is so odd. But yeah, that’s the big thing that I…
Kevin Gardner:
Mm-hmm.
Yeah. I mean, you know, imagine Excel evolving over time instead of it being predictable. It like the, the, the, the, the AI itself just evolves and it just changes. And that’s kind of what we run into. mean, we run into the same problem in development as well. It’s like you get used to something being predictable, like being able to, get it to, to build you out a module or build you out this.
And then the next time you check it, builds it, but then it changes something. And so, you know, there’s a level of predictability, the same with development that we expect. We kind of learned when it comes to AI to rely on it for what it is and expect the unexpected because it’s changing constantly. If you look at it from the perspective of it being an adventure, that’s great. kind of makes it, it makes it interesting with me as a manager.
Dane Dickerson:
Mm-hmm.
Kevin Gardner:
Speaking with developers and encouraging them to use these tools When they have negative interactions with them and in other words slows down their development process or ⁓ They feel like it’s taking more time to train the thing than to actually do it but you kind of have to make the judgment call yourself if you feel like this is going to be something that I’m going to need to do like for example, let’s say I have a I have a client who needs to do a product import and they need to to correlate multiple spreadsheets together to get all the data You would probably as a person you would probably spend three to four hours working on that and An AI platform if you would have whichever one you happen to be utilizing could probably get that done effectively in about Ten minutes with the right prompts but the kick the key is the right prompts is getting the right prompts and that’s where the training comes in because
Dane Dickerson:
Mm-hmm.
Okay.
Kevin Gardner:
You know, we kind of think of it as we’re training AI, we’re training AI to do better, we’re putting in steering files, but it’s the same as when you’re communicating with a new resource or a new junior developer, you also have to learn how to talk to them. It’s no different with AI, but the difference here is, that junior developer is changing all the time. So you have to learn the new way to talk to them. And it’s so it’s.
Dane Dickerson:
Mm-hmm.
huh.
Kevin Gardner:
Not kind of looping back to what I said earlier about needing to be technical. It’s not even necessarily that you need to be technical to know how to read a markdown file. You have to be technical to know how to just talk to the thing. ⁓ Not even with technical jargon, but just like you have to always keep an open mind and be able to, know, be amenable to actually speaking differently. With the day with your prompts and there is there are full disciplines and full courses you can take about how to write the best prompts to get the best results for things. And you find more than anything that you are writing narratives rather than writing code. Yeah.
Dane Dickerson:
Right.
And the auto-complete-ish feature of it is also such that understanding, if I approach this prompt with very biased language and it’s clear what I think about the problem, that’s probably going to be reflected back in the response. And this is not a neutral third party or anything like it. This is like a dumber reflection of myself too.
Kevin Gardner:
Mm-hmm.
Dane Dickerson:
Yeah, gets back to, okay, this is person-like, but it’s not a person. It has problems that are similar to junior developers and not similar, because it’s never really going to learn and the models are changing. But at the same time, when it’s focused the right way, we can get really good results out of it. I think something that’s positioned us well for this becoming like AI entering business processes, which that’s really been the last year, is this stuff getting useful enough to be in work delivery. But prior to that, we took a pretty big stand as a platform agnostic company in our approach to different technologies we work with, saying we will advocate for and learn what the best technologies are to solve these problems versus we rely on a strict set of them.
So we already had a lot of these steps in our process for building sites and doing migrations that are outcomes focused and apply to any technology stack, such that bringing different tech into our own stack for development, we know what we need to be looking for to see if that was effective or not.
What do each of you think, I’m curious, is something that we will be doing in a completely different way this time of year from now, without these tools?
Eric Leslie:
I think building upon, as we continue to understand where it is helpful, I Dan, your team’s done a great job with that from a non-developer standpoint. But again, it’s just identifying the strengths and weaknesses. But I think that for project managers, it’s going to adjust the workflow that we work in, again, whether it’s a wizard-driven or IDE or command prompts, whatever the…
The way that we’re interfacing is, I think that it is going to, I don’t want to say automate because that’s not it, but you know, okay, I’m feeding it the information that I know. It could be from a meeting. It could be from a conversation internally. It could be from Slack, whatever the source is. Okay, now take this information and tell me what tickets need to be created. go and create those tickets. Ask me what you need to know. I’ll get that for you or point you in the direction.
As an example, I’ve been working with Paul, one of our architects and lead developers here at Human Element, and we bought a remarkable tablet. So I’m in the process right now of migrating to this remarkable tablet where I can take my notes throughout the day that I don’t take digitally. And then at the end of the day, I OK, AI hears my notes from the day, tell me what tickets I need to write, or things like that. So I think that within the next year,
It’ll be very much taking all these different sources of information and say, okay, just go identify what we need to do to continue work moving along and get it ready for the other teams to act on.
Kevin Gardner:
Yeah, a couple of them, and I think this first one is one that you use actively, Dane, is encouraging people just to talk to AI. Just use the tools that you have available to you. And if you’re comfortable doing it, if you just want to do what I like to call a brain dump, where you just had a meeting and you need to go to another meeting and you give yourself five minutes at the end of the meeting so that you can just brain dump your stuff. You can use remarkable tablets like Eric is talking about where you just write it down.
But just being able to just like have a tool that you can just literally use the same mic you just were in the meeting for to just talk and dump into AI and have it give you some sort of summary. We have a lot of platforms that do that already. Now we have, know, Zoom has its own transcribe and things like that. But in the event that you’re, that it’s not available to you or you need it faster, being able to make that part of your workflow, your personal workflow to do that.
And it’s not only going to be valuable, that is not only valuable for people who are in a managerial or in a strategic setting where that is more high stakes, that’s also valuable even for developers who only have a couple of meetings a day or maybe a couple of meetings a week. That’s incredibly valuable for them because they can drop out of the meeting and go.
Yeah, know, Eric told me I needed to do this thing. Dane told me I needed to do this thing. Kevin was talking about this one thing. We were talking about this and we needed to do this. You can just kind of brain dump on that. That’s probably probably one of the bigger ones. I know Dana, you’ve talked about utilizing that as well, just in kind of a workflow.
Dane Dickerson:
Yeah, I mean, that example is almost, I that doesn’t feel like a year away. That feels like, at least for us as a company, that’s pretty much how we develop content now is it is usually conversation based either between people or it’s me talking into my phone for a half hour and then doing.
Kevin Gardner:
Yeah. Yeah.
Yeah. Maybe, okay, maybe I should have prefaced that by saying getting folks comfortable with doing that. I feel is a little ways away because I don’t feel people are comfortable sharing as much with platforms like that. ⁓ and just sharing their brain dumps. It’s more like this is the curated things the same as I’m writing code or I’m writing this XML or I’m writing this whatever. I’m just going to share that.
Dane Dickerson:
Yes. Right.
Uh-huh.
Kevin Gardner:
And then that’s and that’s it. But instead of just like just talking to it, I think that we’re going to find that people get more and more comfortable doing that within the year.
Dane Dickerson:
Yeah, gosh, one of the funniest things to watch happen as a group, and we have done this too, is to have a Zoom meeting full of people and one person a screen sharing a chat bot, and we all have a big debate about what to type in. And that is the opposite of what is most effective most of the time. It’s usually best to provide lots of information, see what sticks, and start to get used to how to interact with them successfully.
Kevin Gardner:
Yes.
I didn’t want to mention one other thing. I know I mentioned just a couple here One of the thing that we don’t really have a representative here to talk about this but a quality assurance QA is one thing that I see it within the coming year. That’s going to change drastically It’s likely changing drastically with other agencies and other entities that work alongside us in the industry and that we’re finding that quality assurance in general is just more effective when you can find the right platform that meets your needs and that you can set up automated test suites and have AI build it. There’s a lot of ⁓ platforms out there that are getting there. It’s the same as with AI and our usage over here. There are a lot of platforms that are getting close and getting there.
Dane Dickerson:
Mm-hmm.
Kevin Gardner:
But finding one that ⁓fits your needs completely has been a challenge for us as an organization as well. But I do feel within the year, we’re likely going to be looking at it from the perspective less of, we still have a place for manual testing. I think that’s always going to be a thing, but I think that automated AI driven testing, that’s just something that just happens like automatically, you don’t really have to.
Dane Dickerson:
Okay.
Kevin Gardner:
Think about it. You don’t gotta think about it till it fails, right? And it just happens whenever we do deployments, whenever we do updates, things like that. You can just depend on that being a thing. One of the biggest concerns we’ve had about that is in whatever approach you happen to do, if it’s not curated well, then that’s gonna cause a headache for delivery because that’s gonna create a lot of bugs. It’s gonna create a lot of things like that. So it’s always gonna be a challenge, whichever approach you go with with that.
But I think that that is going to be something that we’re going to be looking at. We’re actually actively looking at it right now, know, but actually implementing and getting it where it needs to be to be the de facto way that we do quality assurance. I think it’s going to be something we’re going to see within a year.
Dane Dickerson:
Yeah, something I think about with software testing, and I knew a few people who went into the worst of all software testing, which is video game quality assurance, ⁓ where what you’re frequently asked to do there is to do something mundane and repetitive that is likely to cause some kind of code issue. You’re going to run out of memory. We’re going to cause bad things to happen even though you’re using the software in the strangest way possible. But that’s how you discover bugs. I think we might head towards a place where QA across all of the software industry starts to need to be able to account for these more edge cases in automated testing suites, which for us is pretty exciting because we know for especially when you’ve got a client with a niche B2B site, limited audience, that is a pretty safe environment. So we know if we have done extremely extensive testing to cover edge cases, that gives us lot of confidence in that type of environment with that set of customers. In a way that’s, yeah, ultimately you don’t want to have to think that hard.
About QA: You want really simple, repeatable tests to make sure nothing has broken.
Eric Leslie:
And kind of weeding through what’s valid. One of the examples that came up was the bug ticket was about a link in the footer and an AI tool we were using for QA spit back 47 test cases just to test the link in the footer. And it’s like, okay, just like any AI tool, how do you weed through the amount of data that’s returned sometimes and realize what’s useful, what’s not?
Dane Dickerson:
Yeah.
Kevin Gardner:
And not only that, it kind of goes back to what I was saying earlier about AI, it’s always changing, right? So, you know, a test that’s valid that you wrote now, is it still gonna be valid? Is it gonna self-heal enough to be able to update with the AI logic changes? That’s one of the things that we’re running into as well when we’re looking at these tests, like older test cases in those older suites.
Dane Dickerson:
Yeah.
Mm-hmm.
Kevin Gardner:
They don’t work as well as they used to because the logic is changed so You know kind of dependent upon the platform to be able to self-heal and fix those test cases and things like that they tend to look at self-healing as You know something on the the entity that’s being tested has changed like Dane’s example of something in the game changed, right? So we’re gonna test but what happens if the test itself is out of date? You know that sort of thing is something that we’re running into as well
Dane Dickerson:
Mm-hmm.
Yeah, if being a QA person becomes a job of I maintain test cases, like that is one way for that to go really. Like your focus is almost more on, okay, these are all of the tests that returned back today and here’s what failed. And the job is to go in and understand why that is if the test is broken, the site’s broken.
Eric Leslie:
And then even taking that one step further from a business standpoint, obviously, we’re a business at Human Elements. if checking one link causes 47 test cases, can you really charge by test case, for instance? So what’s fair to our clients? What’s fair to us to pay for those tools, but also be realistic?
Kevin Gardner:
Right.
Right.
Dane Dickerson:
It’s something I think will be different for our group, for strategists and consultants and marketing in a year’s time is that we like to think of features. We like to think of site features that would be valuable for clients or good for their audience. And something we may be doing in a year is more often demonstrating these capabilities or showing this concept by doing proofs of concept because that is something we can show the technical feasibility of the idea really rapidly. Now that code is usually bad. is like, it barely works, but is enough to show that the concept is technically possible.
And for me always, if I can show what I mean, for this kind of thing that’s gonna work better than any PowerPoint deck or really compelling sales call and it should be believed better. It is proving the concept. And something that I like about having my team using more of the development tools, not because we are great at development.
But it gives us a lot better understanding of interfacing with development, what we go through here. And having some of those tools to know even what’s a good question to ask a developer about this, and what’s a bad question to ask them, makes us that much smarter, gets us to making better decisions.
Kevin Gardner:
Mm-hmm.
That’s one thing I can say that’s probably been a really good bonus of AI is it is actually, at the very least, I can say from the client strategy and the development perspective, it’s brought us closer together. You know, we’ve always…
Dane Dickerson:
no.
Kevin Gardner:
It’s really easy when you’re in a business, regardless of what the size, to really silo knowledge basis, the silo roles, right? And it’d be like, you go to this person for all the things, you go to this person or this role for all the things, this role for all the things. And it’s really easy to do that. But especially when you’re a smaller organization like we are, it’s really good to be able to have people who can wear multiple hats because you have to, right? It’s just the necessity of our business. then having, you know, Dane’s group having the ability to be able to just talk more effectively with a developer because they understand it’s really easy to do when you understand what a developer’s doing. You have a better idea of how the platform works or the code works or the thing works.
That you can ask the right questions. And you get a better response generally from a developer when it comes to asking the right questions. You ask a silly question and it’s like, it’s not even worth my time, you know, kind of thing. we don’t really have developers do that. But I’m just saying you’re gonna have a more effective level of communication between what used to be considered more like,
You stay over here, we’ll stay over here and do our thing. It’s more of a collaborative effort kind of thing now. So I’ve noticed that just the ability to communicate and the ability to be able to do work and talk with developers between, especially like a strategist in marketing and development has gone up a lot. It’s been a lot more effective.
Dane Dickerson:
Mm-hmm.
Yeah, something I can share as an example of maybe something that is clear to us now, but in the past I’ve seen, not a human element, but other places, is one easy ask for a developer to turn into a hard ask, and the person asking the question has no idea that they’ve done that. And specifically with the difference between, could I get an export of this data out of the site?
Turning into, well, could you just build me a dashboard? But one of those, like one of those is a few shell commands to kick out a database and then you could do whatever with it. And the other is building a visual designed, lots of opinions to go into it software product. And just knowing that we can do the first easy and the second’s gonna be a lot harder.
Kevin Gardner:
Mm-hmm.
Yes.
Dane Dickerson:
I mean, yeah, our developers are all very nice, but they are right to roll their eyes at a data request into a design app request.
Kevin Gardner:
Well, that’s the one of the things that kind of feeds into that is that we, yeah, developers can be really opinionated on how we feel like development processes should be, but it’s because it’s all lessons learned. We’re essentially bringing AI into something and we know that we can turbocharge what folks that are not considered developers can do, which is amazing. We still have to have developmental standards with things because it’s really easy for things to go off the rails.
Dane Dickerson:
Mm-hmm.
Mm-hmm.
Kevin Gardner:
Everyone at this point in time that is likely listening to this knows what AI slop is And we don’t want no one wants to be known as the people who who generate AI slop, right? So maintaining developmental processes maintaining You know code reviews and everything that you could think of that would be in part of the normal process that yes It does slow the process down. It does make things a little slower, but it makes things more deliberate
Dane Dickerson:
No.
Kevin Gardner:
We were more deliberate with our processes. We’ve been in that pain point already. We fixed it fairly quickly, from an organizational standpoint. it’s overall just making sure that you use AI for what it’s good for. And if you feel like it’s an opportunity, you always kind of need to keep it in the back of your mind. It’s like, this is doing it fast, but like Dane’s example of like,
I could do a couple of command line things on the command line and maybe even tell you how to do it. Or we could build a whole dashboard that’s going to require development processes, scoping, phases and all stuff like that. What’s better? know, what’s going to be better in the end?
Dane Dickerson:
Hmm.
And like, answer is, it depends. Yeah. Yeah.
Kevin Gardner:
But we have the power to do that now, right?
Because before it would have been like, all right, Eric, so we’re, we kind of want to make this dashboard, right? And we want to work on it as an internal project within here, within human element. Can you make us a ticket? And he’s like, all right, I’ll make y’all a ticket. And then it just goes and sits in purgatory for a while because no one has any time or we don’t have any push to do it.
Eric Leslie:
One thing that might be interesting to spend a minute on is what do we see as blockers to the next year goals that we set or what our vision is? Like what are some current blockers that we run into?
But when you have people who are excited to be able to do these things, makes a difference. And being able to turbocharge like Dane’s team, for example, to be able to do this stuff, they’re excited to do projects like that that could potentially turn from internal to billable projects, the things that we could potentially sell. It’s a huge deal. And now that we’ve got them working on direct billable things that we could put in marketplaces and things like that.
It’s been something that’s been growing a lot within the company.
Dane Dickerson:
So it’s a challenge to be ahead of it too, because we are in an industry. We are not the only smart people in eCommerce development or site development or marketing. But I like to think that we have a pretty good sense of what is genuinely useful and generally makes a difference for clients. So it’s been fun for us. And I think staying, yeah, staying ahead of that and staying at the edge of like, is the best we can get out of the tools we have today is gonna keep doing well for us in the long term. Any last words, either of you?
Kevin Gardner:
I can start us off on that one. One of the things that we as an organization find concerning is just data sharing within AI in general, like the privacy of data. Obviously, we want to use AI, but at the same time, as an organization, we need better guarantees that our data is not going to be used to train other models and be made available to other competitors or entities that we don’t want to see our data, that we don’t want to have them to have access to our data or our work that we put into the mix. it’s that balance between doing our collective best as humans to train our chosen AI platform or platforms to make them better for us, but at the same time, I guess, selfishly not enabling our competitors to be better at it as well, because that’s just the nature of business, right?
And not only that, but just general privacy concerns, right? For example, ⁓ we have a hard hesitation right now with using AI for anything HR related, for example. There are HR…
Specific ai platforms that you can use and we have our own platform that we’ve chosen as well But we don’t really want to do that because we don’t know Who in the company could potentially see these things and who could see this? So that’s that’s probably I mean it may not be the biggest challenge that we have Well, they are right now, but I think that that is an emerging issue the same as Some of these other larger companies that we’re going to run into in the coming year
As being a very serious concern that we’re gonna have to button up legally to make sure that that we’re that we’re in compliance across the board
Eric Leslie:
You know, kind of to build off of what you said, Kevin, not only is it our own internal policies and rules and what tools we can use, but, know, it’s also being good stewards to our clients. You know, they all have their own policies and that kind of even restricts what we can do in some cases, you know, being aware of, okay, these clients are fine with using AI tools. These ones are not. And so you’ve got to, you kind of have to have these choose your own adventure. Okay. I can use this tool for this client, but not this client for instance. So.
Kevin Gardner:
Mm-hmm.
Eric Leslie:
For me, that’s one of my biggest limiters or challenges at the moment is just I can’t fully leverage some of our tools because, okay, 80 % of my clients have policies against AI usage in some regards.
Dane Dickerson:
Yeah, there’s a similarity there between that and a lot of what’s been difficult with autonomous cars as a forerunner example is there are situations where there’s almost more concern about accountability than quality. So you can say, this AI tool when used for HR, it produces better outcomes 80 % of the time that could be statistically true. It’s a little hard to quantify. But the 20 % it doesn’t, who’s responsible for that? Is it the tool? Is it the person using it? Is it the software vendor? And HR or anything that’s legal implicated is all about accountability, much more so than quality. So I would agree there. That is a huge problem to solve is
Kevin Gardner:
Right?
Dane Dickerson:
Where in these kinds of decision making processes, can you introduce these tools that preserves still accountability and authority within them? Something that I, I guess, worry about or could anticipate being an issue in a year is we have so much software that is being written at the moment and published at the moment that it’s not really clear how it’s come together.
And some of these are cool. One that I put in the blog post we put out about the same topic is one that’s offline files and will index all of your external drives and keep that index on your local machine to browse through so you know what drive you need to plug in if you’re looking for a file. Really neat. I am suspicious of how this got coded because it seemed to pop up out of nowhere from a first-time developer.
Kevin Gardner:
Mm-hmm.
Dane Dickerson:
And it does seem to work really well, is this reliable, consistent software? And for every one piece like this that seems to work well in practice, how many pieces of software floating out there that are going to be really difficult to work with in a year or two? Or new prospects that might come to us that vibe code at their redesign. What are those code bases going to look like, and what kind of problems can we predict there?
Kevin Gardner:
Mm-hmm. Yep.
Yeah.
One thing I’ve noticed, and I don’t know how much you guys have followed this, but we actually have AI platforms and AI entities that are being released now that specifically exist to take advantage of vulnerabilities. So like tying into what you just said, Dane, that’s going to be a huge issue where you have now. Not only are AI entities themselves there to help us, but they’re there to also take down competitors and also just there to poke holes in an existing platform. So, there’s a big concern in our sphere of, we have quite a few e-commerce platforms that we support as a company. Some of them, not gonna name any names, some of them are a little bit more slow and more prone to having issues and letting slower to respond to fixing them and are more laborious when it comes to actually getting those fixes out there. And you’re never gonna have AI that it doesn’t sleep, doesn’t eat, you just feed it a data center and it’ll run trying to crack code 24 hours a day.
And it’ll find bugs, from a technical perspective, which a lot of this stuff is being driven by technically minded people, from a technical perspective, that is absolutely revolutionary. That’s amazing. But from a business perspective, that’s a nightmare. Because you never know what you’re gonna run into when you come in tomorrow. Right? Yes? Yeah. Like very much so. So you get the instances, Dane, like you were talking about where you know, get this big index of your offline files. Who was reading that?
Dane Dickerson:
Mm-hmm.
I was reading that. Does this write a new log file every time? Is this going to be taking up 100 gigs of my storage in a year? Because there are issues you couldn’t have seen at this point, but you at least hopefully could have if you’re a developer architecting every piece of that system.
Kevin Gardner:
Yeah. Is it going to, is this tool going to go out and scrape this archive this. And then now all of a sudden my Facebook results are changing. Am I at results? Because it’s like, what, how’d I get this? There you go. It’s because it read an index and a very obscure thing that you wrote three years ago.
Dane Dickerson:
Is that why Facebook Marketplace keeps wanting me to buy this geotracker down the street? It’s telling on me.
Kevin Gardner:
Could be.
Dane Dickerson:
All right. Well, thanks both of you for chatting. Nice to talk this through and who knows, maybe we will again in six months and it’ll all be different. We’ll find out.
Kevin Gardner:
for sure.
Eric Leslie:
Very likely.
Kevin Gardner:
Yes.
Dane Dickerson:
Alright, thanks guys.
Eric Leslie:
Thank you.