I Gave AI Full Control of My Blog Design, Big Mistake?

Hey folks, in a previous video we were looking at using chat GPT of all of the tools to be able to help redesign my website and then I showed how that I could take the output from chat GPT, feed that into C-Pilot, get a bit of a visual explainer, if you will, and then basically from there I went off on my own to go let co-pilot run, do its thing, and perfect my website. Now, we're going to look at the website and see what it was able to do over the course of a few days. just kind of mostly running autonomously. We're going to compare that to what I actually gave it as a output from chat GPT in terms of like the the concept like where I want this to get to. We're going to look at the differences, but I'm going to walk through essentially what it was able to do on its own with a little bit of guidance for me along the way. So, just going to flip my screen over here. This is where we had the handoff from Chad GPT. Um, if you haven't watched the previous video, I recommend just going back. You can kind of skim through it if you don't want to watch the whole thing, but I went to chat GBT essentially because I wanted a new navigation bar made for my website. And then ChatGBT was like, "Here, hold my beer." And then made me a whole new sort of uh set of concept images for my my website. And when I saw sort of these pieces coming together, I said, "Holy cow, I definitely want to change what my website looks like." Because even if you know, if you don't like the aesthetic of this, that's totally fine. But for me, I was like, "This is just simply way better than than what I have right now." Um, so what we did was we took this output from chat GPT, went over to Copilot, did a bit of a handoff, and I'll walk you through that in a moment. And then the output which finished roughly I mean I checked this morning for a couple things to close out but this is roughly done uh before I went to bed last night. There's like I said a couple of issues to close out but mostly just housekeeping. And so it's not the exact same. Obviously this is actually my face whereas on this one we had um I don't know someone completely random but there are definitely some differences. We'll talk about that. And overall, I think I'm definitely much happier with how this looks compared to how it used to look. Um, I have, you know, it went through like all of these different pages. Uh, made them this newer theme. Let's see. Like this navigation is way better than what I had before. But this was really what I was after because I used to have this title bar, uh, sorry, this navigation bar that had so many things on it, it was useless. So, I got what I wanted plus a little bit more coming out of this. And even, you know, if you're on my website, you can check out my YouTube videos directly on here across my other channels as well. But it's all sort of done and uh rethemed. So, with that said, I'm just going to walk us over to Copilot. And so, I actually used another session to summarize the previous session that did all the work. And the reason I didn't put a video together uh for all of this is that walking through this on fast forward would probably put you to sleep and no one's going to want to watch that. So, I figured the best thing that I could do at least is make it so that I get this working. I get it to a point where we can talk about it and then explain the steps I was taking so that you know you can try applying this in your workflow. Maybe there's things that I'm doing where you're like, "Obviously, Nick, I wouldn't do that." Or things that I'm doing where you're thinking, "Hey, like maybe that is something I should try." But let's dive into what's on the screen here. So, what I have is the summarization of another co-pilot session, like I said. And if you watch the other video, what we started with in or I guess what we ended with that video and what I started on my own was this handoff process. So, I took the information from ChatGBT. I had uh some design briefs that came out of it and then I gave it all of the screenshots that came out of ChatGBT. Um I I don't know if everyone remembers like I guess like a couple years ago ChatGBT making images and we'd have like you know uh these Will Smith spaghetti eating images and now I feel like chat GBT does a tremendous job making images and I I just really like using it for iterating on design work. um seems to just work really well for for me at least. So that's my take. But I had the design briefs and then these concept images and then what I did was I gave that to co-pilot and I said hey like I want you to read through this like tell me your interpretation right tell it back to me and then from there we ended up getting this visual explainer. So, the visual explainer is uh and I will make other videos on this, but I have a a skill that I put together. I'll show you how to make the skill and you can take this kind of concept and use it for anything you want. But the idea is that if I want something explained to me, um instead of just having co-pilot list out a bunch of stuff, I'm like, if I have a visual, it helps me a lot. So, we had a visual explainer on how it's going to do the work. And then the very next thing I did was have it make a road map. Okay, so this is not something I always do, but on this particular type of work, I thought that this would be handy. Uh, and the idea is that I just basically allow it to go make a bunch of GitHub issues. This is just my own private repository. Anyway, so it goes and makes these issues um, and it breaks them down into uh, decomposible pieces. This isn't perfect and I will talk about this as we kind of walk through what it was doing in this work. But one of the things where it really breaks down uh I already used the word breakdown but uh where it falls apart I guess is on really long running tasks and I think that this is a really good parallel with software engineering in general right so my my thinking here is that if you've done any type of you know larger projects in your software engineering career when we talk about planning right like I think upfront we can come up with plans and we should we should try to think about plans we should try to think about where we want to get to uh from where we're at. But ultimately, the longer the time horizon, the more complexity there is in whatever plan, the more that things are probably going to fall apart. This is why uh you'll see all these battles about like, you know, waterfall versus agile and all this kind of stuff. But I think the reality is that whatever you want to do, there needs to be some type of feedback loop because, uh the odds of you, you know, perfectly predicting a plan and executing it uh are slim to none. And so instead of just assuming the plan will always be perfect, assume there needs to be some type of iteration. So in this particular case, uh, Copilot did go break down, you know, a road map on how it was going to deliver things. And I think one of the things that it did that was kind of interesting is that, um, and I haven't seen it do this before for other work I do like this, but not only did it have a road map of the deliverables it was going to land, but it kept a couple of like other tracking issues open. and those tracking issues like you can see this in this summary it's saying uh the intervention great word added three crosscutting requirements so as it was delivering all of these uh different screens that it was going to update it actually said I need three tracking issues for things like my agentic instructions skills and documentation being kept up to date so every time it would land you know I'm going to go update this particular screen like it's going to update my newsletter sponsorship screen then it's going to make sure that across these three items is it keeping instruction skills and documentation up to date uh is it actually measuring the performance because uh if you've tried doing this kind of thing with AI and uh doing website refactors and stuff like that my experience has been that um yes it can do a good job but also sometimes it will completely sabotage like other optimizations and things like that. Uh so um I made sure like hey it's going to measure. So not only is it going to measure before it pushes things live but once it goes live reme-measure right there's other infrastructure that's running out there in the cloud that is not being simulated on my machine or in CI. So go reevaluate it. Same thing with caching and things like that. So has all these issues and then um essentially from there you can see right here it says initially this was not autopilot right so I started this work stream off saying let's get this thing going I want to make sure that it's on the right track before I hand it over for autopilot and again um this isn't the case for everything I do but I think that there is some things in particular where I am very comfortable with letting it run on autopilot like in Full transparency, as I'm recording this video, there are two video games being developed on my desktop, right behind my recording window, and those are being done completely on autopilot. And that's because how I'm developing those looks different. And same thing with my website. There are things that I really do care about and then once I see that it's on the right track, I'm going to let it go on autopilot. So, we started things off making sure that I can get some evidence that it's going to do the right thing. So, um you can see navigation illustrates the feedback loop particularly well. Agent implemented desktop compact navigation. This was the first feature I wanted and then I said I accepted it. Um but the the mobile menus that it was creating were actually um they were wrong. So, it's like it did uh it did the full menu. Maybe I can just actually go back to showing uh the screen here. So, what it was doing was when you went to like this menu or you went to the topics when it was in the mobile view and it was smaller, it was basically trying to jam this whole thing like in this very spread out view. It was trying to do that across um like a very very tiny tiny screen. So, um not going to work. And so I rejected that but said, "Hey, like the desktop stuff's good." Talked about how like I I do want this to look. So we had a bit of a feedback loop, right? More of an interactive session. Um it corrected it, right? I approved it. And then um there was one more set of work and let me pull that back up again. So the other set of work that we had here was that when we look at articles, um that is a newsletter. One sec. this one. So these feature images, what it did was in its design uh or I guess in its its first like refactor of this, the background or feature images, it was putting right here. So it was doing a sort of uh unhindered view of it, but it was sort of like this really weird placement that I wasn't a fan of. And I realize that this one's kind of dark behind, but I would much rather have the image like at the top versus like this like afterthought placement. So I rejected that. And then um we kind of had this back and forth just for a couple of issues. Okay. Um and then you can see here that after I did that and we had a little bit of back and forth on how it should work and how to approach this kind of thing, then I said this. I want you to work on autopilot shipping this in order. And then I gave it from its issue breakdown list um this set here. And the actual work items don't specifically matter, but there was one in particular. You can see it says 216 decomposed uh 218 219 220. This is one where I'm reading what the issue is and it's trying to do like far too many things in one deliverable. It's like we're going to do this screen, then this screen, and then for issue 216, we're just going to do the rest of the screens. And I'm like, "No, it's not. It's I know that's going to fall apart because it's too many things at once." I find when it's running on autopilot, and this is just been my experience, when it's running on autopilot, it can do longunning tasks, but if I try to bucket too many things, even into like a single GitHub issue that it needs to deliver on, it makes a single pull request for that issue with far too many things. And so if I make the GitHub issues smaller, the tendency for it to make smaller pull requests that are more incremental, more manageable, uh seems to align with that. So we have this issue list. Um and then you can see it's recurring execution pattern because this is sort of what we establish by doing it manually is inspect baseline because I need to get uh some performance characteristics, SEO, AEO, GEO, things I care about for my website. um failing regression test and then implement hosted checks rendered review correct merge deploy right then I also mentioned that it needs to do a live audit and that's because there are some things that I cannot simulate properly um whether it's on my machine or in my CI system so hopefully those are catching uh nothing ideally once it's live there's no issues to catch but that means there is at least this check right And for what it's worth, some of the things that slip through are like it goes and runs uh page insights, right, to page speed insights to get some performance characteristics. And it's not a perfect system, right? If you've tried running page speed insights on your website, you'll notice it's never like exactly the same each time. So, there's a couple of discrepancies. And I think overall, like when I get to the end of this, there's only one page that that really had uh a regression. And quite honestly, I'm not even bothered by how much it regressed. So everything was pretty good overall. So um you can see the long session retain continuity through checkpoint records completed work blah blah blah. So it's it's going through it's actually delivering on these items and then because the actual session running long is checkpointing things like that. I'm not using entirely separate sessions. I think different people have different approaches for this kind of thing. I'm personally the kind of person who likes keeping really long running sessions, I like having things organized in my co-pilot sessions that way. Uh so instead of doing sessions completely based on I want this feature made, go build it now new session. Um I approach it like hey you are a worker on this project and so you will or at least in my case I will keep this session open and that means when the work is done or I want to completely pivot I'll compact the conversation explicitly and I mean this is happening anyway but I'll explicitly compact and then I will have it go do some other work and one of the things I I like about this is I I personally find that it seems to gravitate more towards I don't know like the pattern turns that I'm I'm after how I'm approaching building things. And there are absolutely times where regardless of me doing that, it seems like the whole thing collapses anyway. Um and then I kind of have to do a little bit of work to get it back on track. But um for the most part, uh I do like keeping these uh long running sessions and I just compact. So it's going on and delivering all these things. But you can see success was measured at several different levels. And this is going back to these uh if I scroll back up here, these three tracking items, right? So these are things that it's constantly measuring as it's delivering, right? But uh success was measured at three or several different levels. Does it resemble the approved direction still work? Uh existing contract survive. So this is really really important for me. Uh, I do a lot of work to make sure that my website is going to show up in in Google AI search things like that. So, there's a lot of structured information that's important. I don't want it to go say, "Hey, let's go remake this screen to look a particular way and then all of this work that was done to have structured metadata and things like that is just completely gone and wiped away. Um, the change actually live, right? Did it actually complete? Could they could it get tested?" And then this performance uh acceptance which like I said is uh it's monitoring once it's deployed live. So it's doing that. It's actually reporting each time as it's going through. So there's a record of that in the chat. Uh this this is a different session, right? I I mentioned that I had this session go analyze the other session that was doing the work. And so you know you can see it reported mobile median uh fell from 99 to 91 after the redesign, right? like that's not in this session, that's it analyzing the other one. Um, so the point is that that information is recorded in the session and uh the co-pilot was doing that as it was going. So for me that's really important for longunning work is sort of this observability idea. Um I mentioned already that there's some work that I will do on different projects where as the features going like I'm pretty attached and then in this particular case as I'm letting it run and deliver I want to make sure that if something's off I have some data to go back and figure out where it's off why it went off course. So finally um one of the things I wanted to say is that this type of thing uh doesn't work perfectly of course as you might have guessed uh and this is where if if you were to think about where I spend most of my time these days it's kind of in uh in this phase. So, um, to kind of talk with my hands a little bit more, the idea is that I, as much as possible, I'm trying to get to the point where I can set things up how I want, let the agent execute, um, as much as possible give the agent the tools, information, the things that it needs to do to follow a development process in a way that I like without me having to intervene. Like, I in a perfect world would love to be able to say, "Go build this and it does it perfectly." It's not a perfect world. So I think the next most perfect thing that I can get to is let me define something. Hey, go build this. I can have full confidence that it's building it in terms of code, infrastructure, architecture how I want or how I would like if in a perfect world I'm perfect too, right? And then the thing that I want to do is kind of look at the result. I want to interact with the website and say, "Yep, this feels like how I want." or for other projects, products, things like that. I want to use it and say, "Yes, this feels like the thing I wanted." Right? I don't want to have to babysit the agent as it's going. However, that's not the reality we're in. So, in this case, what happens is that there's a bunch of things that kind of start going wrong along the way. Uh, so some of you can see like the weaknesses are were also substantial, right? So, yes, it's iterating on this, but there's some things it's doing that were just completely wasteful. This work took several days of co-pilot running and it shouldn't have had to. I mean that also includes things like CI time which should have been optimized but you know custom asset retention machinery exceeded the demonstrated need right so it started building things that were completely overengineered and I happened to catch it doing something because I happened to be checking in on it and I was like no no no don't do that like there's literally something that exists for this stop building something completely off the rails so yep I had to catch it uh from from doing some stupid stuff. Says the agent sometimes treated cued CI or unavailable production identities as stronger blockers than the evidence justified. So part of why this took so long was because I do a lot of my CI uh off of GitHub actions uh hosted and actually on some local hardware. There was an issue. There was an issue where I lost CI. There was an issue where uh even when CI was working, I have runners that scale up and down. And so it was like, oh, there's there's no runners, so I guess I can't continue. But there's no runners because the runners automatically scale down. When you ask for them, they scale up. So sort of, you know, not giving the agent enough information to understand what is available. And so as a result, it got blocked. So I had to check in and go, oh, you know, you sat here for hours doing nothing when you could have been actually working, right? Uh there were some cases where it's not for code review. So for this work stream uh this was something where I said hey like I want to review the finished product as it's landing. I'm not going to sit there and review your code. And so on like I said on some projects there's absolutely things like this where I'm meticulously going through. In some cases like this I'm not too concerned. And so I'm not going to sit there and go read through every detail on the code review. And as a result uh in some cases it just sat. In other cases it merged the PRs through fine and kept going but in some cases it just blocked itself even running on autopilot. Um repeated validation and correction consumed almost a day. So a lot of time lost in that. And this is the kind of thing where uh what's a good example of this? Let's say for some performance analysis, it goes and runs page speed insights and let's say the score before was 95 and now it gets a 91. It's like, oh, it regressed, right? Like what what is the perfect threshold that we should put? Because if I were to generalize this, I might say, hey, you know, don't let it regress by more than X%. Um, and in order to get that, I need you to do X number of runs to get some some median on that or maybe I want to talk about percentiles. I don't know what the perfect solution is for it. So, uh, that's definitely something that I see is that when it's trying to optimize or keep things within bounds, uh, that can really throw the agent off from like going down a rabbit hole. Um, I noticed this a lot with uh CI optimizations, which was uh less of a concern on this one. But if I go ask an agent to optimize some of the CI delivery, you know, it could have one one run where the node was a little bit overloaded and it thinks like every optimization it did needs to be tossed out. So sometimes there's just like this uh what I would call like brittle behavior that changes direction too much. So a lot of uh this was uh wasting time. This was at the end. Uh so yesterday actually I noticed that this was happening a lot because I checked in there was like I think you know one issue left that it really needed to deliver on and I said hey like it's been hours what's going on and it was basically just you know consuming tons of time on this and that's where at some point as I was getting frustrated I was like hey I'm giving you an hour deadline to go ship the rest of this right like you you have been wasting time. You just told me that you were wasting time, so you have an hour to go do it. Um, and then it didn't. It still couldn't do it. Um, so it's like this this is where it's just kind of falling apart because it's not a perfect system. And uh like I said, this is where I am spending a lot more time uh personally trying to invest into like how I'm using agents uh in flows like this where I'm trying to be very hands-off like what is the tradeoff that I'm making for doing that. So in the end I think that it did a pretty good job. If we go back to um this website here, let's get this up. So it's not perfect, right? There's a there are a bunch of differences. So, if we go to the homepage, you can see like this background is still the sameish that I had before, but this background has like a nice kind of gradient. This this image with some other dude on it I probably would want to keep myself, I think, for my own website. If I jump back here, these cards look a little bit different, right? On these cards, the image goes right to the top. On these cards, um I don't know. There's like there's not even an image, but they have these cool like I don't know like gradient little titles on them which I think is pretty cool. Um it did put the topic areas which is great. So like that is something that it carried over. I think that's really cool to have access to that right on the main screen. But overall like this main screen I think there's just this opportunity for a little bit more like polish I guess. But if I wanted like these cards to have a little bit more of that kind of uh you know title or like banner kind of area that's styled like this. I think that would be not too hard to do. We start taking up a lot more space though. The other thing is these tags, right? If you look at the styling of these tags here compared to jumping over here, these are little bubbles. Like these these look nicer in my opinion. So maybe that's something I'll do is have um you know co-pilot go back through now that it's done all of this work across the whole website like hey every time you have a little tag control style it instead to go look like one of these. So overall I think it did pretty good. If I jump to some other sections like uh you can see this sort of background image right this is consistent across a lot of what it was showing. And I also let's see. So that was what is this one? This is the archive. So if I go to uh articles, right? This is the same kind of screen. So again, the background. Mine's just kind of boring and and dark, but this one is more stylized. I do think that's pretty cool. This here for the Oh, it's behind my head. Sorry. Uh this one here for the the newsletter, right? again kind of a little bit boring in the background whereas this one is more styled. So I think there's some opportunity for more polish but overall the the like sort of UI architecture and layout I think is just significantly uh more improved compared to where it was. So I'm I'm pretty happy with how Copilot went through all of this. Um it's not perfect. Uh this is the kind of thing where once it was going I could be pretty handsoff and get to an end result that I'm happy with. So, I just wanted to walk you through like how I approach doing that. And then what I'm going to do is show another video uh after where I'm going to use a different product that's not uh my website and show you how I have an existing UI. I'm going to follow a very similar approach, but I'm going to try showing it end to end. So, that one's going to be a longer video and when that's done, I will link it right up here and you can check that out. So, thanks for watching and I will see you in the next video. Take care.