We recently were charged with the task of reindexing a Magento site, specifically the URL rewrites of an entire store. URL rewrites help Magento take ‘pretty URLs’ and get the correct data with them eliminating ugly query strings in your URLs much like WordPress’ pretty permalinks.

The problem:

This specific task is that the store we were reindexing was 771,000 product skus large. *The store hasn’t successfully reindexed since October 2010*

A couple points that I found to help us figure this out:

1. Make sure you don’t have store views in your Magento that you don’t need (for example, Magento sample data comes with French and German store views). Extra store views create additional rewrites to be written to the table, which over 771,000 SKUs equal about 1.5 million URLs per store. We had around 4.9 million URL entries with the three store views and got it down to around 1.5 million with a single store view.
2. I found it helpful to truncate the current ‘core_url_rewrite’ table in the database. This allowed me to monitor the process better; if you have a large number of rows in that table, you may not see any change in the process for a very long time, until it exceeds the point where it is stuck at. This happens because the reindex process starts from the beginning everytime you run it. If you truncate the table you can see how many url rewrites are being added and how quickly. NOTE: You should probably back up that table before you truncate it.
3. Optimize you server settings. I started by increasing the run time to 64000 (and yes, that is about 17 hours…) I also maxed out our memory with everything I could which on our set up was 3584 (or 3.5 GB). We need as much speed as we can get, 1.5 million rows even at 1 row every 0.03 seconds is still around 45000 seconds of run time 12 hours.
4. Run it through command line via ssh, it always seemed to run better that way, and only reindex the one index you need. (catalog_url).

*Note our hosting provider didn’t allow for SSH to stay logged in for more than 24 hours; we got around this by running the process through cron*

So with all of those steps it was now time to test what we could do. Running it through cron allowed it to run for more than a day straight, but as it went it got to the point where it was just going too slow to make it in the 17 hours. So I bumped it to 640000 seconds ~ 177 hours (or 7.35 days), and crossed my fingers. A week later I got the email — it had exceed its max execution time. What a bummer, I’d tried so hard to make this work adjusting server variables, monitoring rates of the process, running it through SSH and eventually even cron.

Conclusion

I knew at that point it was time to give up, this size of a catalog was too large to feasibly execute this. However, through the steps above you could easily get Magento to reindex with a more manageable catalog size (say anywhere lower than 500,000 SKUs) , but 771000 SKUs is just too large.

Responsiveness is what it sounds like — your website is responsive according to the device it’s being viewed on. This is done through applying common width standards to stylesheets, and then making design and programming tweaks for each size from there. For you, it means that you can have a mobile website without having to start completely from scratch.

So what are some of the common tweaks made when making a responsive website? It starts at the top, much like a desktop website. We take a close look at the navigation and how it’s presented, how photos are presented, and how any interactive elements are laid out. If your site has a full music player, much like the Drew Nelson site does, you might want to present that differently when viewing the site on a phone. You might want bigger buttons, the ability to stream to an AirPlay device, and tracks listed out instead of all in one player.

You might want your photos to take up the full width of your phone, and do it without uploading three different photo sizes to your website manager. And the most common thing? Readable text — a regular size on your computer, and a larger size on your phone so it’s easy to read.

And that’s just the beginning. Responsiveness allows us to do our part to make your website friendly to any viewer, no matter if they’re viewing it on an iPhone, Android device or laptop. It’s all a part of what makes the web exciting, and why we’re proud to now offer it to our clients.

1. Who’s your audience, and how often are they coming to your site? If you’re running a industry news site, obviously updates will be expected often, if not several times a day. But if you just have a corporate “marketing” site, those updates are going to be less frequent.

2. What are your competitors doing? This speaks for itself…if you have competitors who are blogging or updating their site every week, you should strive to do that too.

3. How much is too much? Writing a news post or running an update everyday can not only get tedious for you — it can be content overload for your clients.

There’s no glorified number that states you should update your website “this many” days a week. Do what’s best for you, your company, and your clientele.

But if you don’t update often enough, it can hurt you in several ways you might not realize. First off, Google and other search engines base part of their rankings on how often a site is updated. Fresh content is always good, especially if it connects you with other sites on the web. Secondly, if you don’t update frequently enough, you’re going to lose some of your audience. By putting a blog on your website, your customers are expecting frequent updates. If they don’t get those updates, they’ll stop following your blog, and won’t visit your site as frequently.

So there you have it. While there’s no right answer to how often a site’s content should be updated, it’s generally accepted that you update frequently, and with reason. Don’t update your content if you have no content. Rather, update when you have something to share with your followers that you deem important for them to know about. It’ll help you in the long run with your search rankings, and further your website in your industry.

You’ll find several of the same features as before, but with little twists. We have brand new projects in our Folio section, and you can now see more images related to each project to give you more of a feel for our process and how we met our project goals.

And, if you’re reading this, you’ve already discovered that we have a brand new blog. It’s our hope to write not only about our achievements as a company, but also about industry standards and tools you can use in regards to your brand’s online presence.

So take a look around, and drop us a note to tell us what you think.

You can learn more about our design and process with RSO in our Folio section.

I found this article on Spyre Studios, and thought it applied perfectly to Mindutopia’s core values in regards to design. Spyre Studios touches on five critical design lessons they picked up from Pixar, which can be applied directly to our thinking as a company when we’re coming up with a new website design:

1) Quantity over quality: During our design process, we want the big things to really shine. Clutter isn’t necessary, and it’s not necessary to tell everyone every little detail in regards to your business.

2) Details matter: When designing a website, 99 percent of the time we follow a grid. Why? It makes everything line up and just seem like it belongs where it is. So what about the 1 percent? Well, those are for the extra special sites — the ones that really get away with breaking the rules by having an abstract layout. That doesn’t work for most businesses we work for, but there are a rare few that go against the grind. When designing, we always take into consideration where certain elements are placed, and what’s most pleasing on the eyes.

3) Don’t go at it alone: While we only have one designer on staff, that doesn’t mean everyone else isn’t involved. Every website is a team effort, whether it be the design process or the development process. In this instance, designing requires that everything works together, from beginning to end. Sometimes reality prevails, when things are just not possible, but most of the time, a happy medium can be met and design can stay at the forefront.

4) Have soul: We don’t want our websites to just be there, sitting on a server. We want them to shout from the rooftops, and be able to draw in a reader like no other website can. Just because a website hits all the technical details right doesn’t mean it’s a great website. It’s gotta have something special.

5) Have fun: We couldn’t agree more…what fun is design without enjoying it a little?