A Camel Thing

We've Upgraded the "Your Price Watches" Page

camelytics — Tue, 17 Jan 2012 20:46:00 -0800

First of all: sorting is back. Please let us know what you think in the comments of this blog post.

The main benefit of the redesign of this page should be quite obvious: where we formerly had a list of price watches that could contain the same product many times in a row and often spread across multiple pages, now you should see one row per product with the individual price watches contained therein. This shortens your price watch list and makes it easier to get a global view of all price watches for each product.

Similar to our search redesign, we’ve added inline price watch creation and editing. Wherever you see a grey dash in the Desired Price column, simply click and you will be shown a box for entering your desired price; click the green check mark or hit enter on your keyboard and the price watch will be created. Editing them is done the same way: click the desired price of the price watch you want to edit, and you’ll be given a text box for editing your desired price. We hope this makes it easier to quickly manage your price watches.

Wishlist Synchronization Fixed

camelytics — Mon, 16 Jan 2012 16:02:16 -0800

Over the last couple of weeks, we’ve had some trouble getting wishlist syncing to work reliably. This is due to the methods through which Amazon forces us to collect the items in wishlists.

Thankfully, we believe we’ve found a solution and expect wishlists to be synchronizing every other day as they used to. Sorry about the inconvenience!

How Our Price Checking System Works

camelytics — Wed, 21 Dec 2011 02:53:00 -0800

Given that we regularly answer questions like, “How often does product X have its prices checked?”, we think it’s a good idea to regularly post statistics about our price checking system. This is the first post in that series.

Price Checking Frequency

This depends on a number of things, the first of which is the retailer. Newegg and Best Buy are checked every few hours, and zZounds and Backcountry are checked a couple of times per day. However, prices at these retailers don’t fluctuate that often, so we don’t always see price changes every time we check (even though we check all of the products they make available to us).

Amazon is a different story: in many cases, their prices appear to change as often as we can check them, and their product catalog is so massive that we do not have a complete copy of it. Our database is populated by users searching our site and by users of our browser extensions. Even so, we monitor the prices of about 6 million Amazon products across 9 different locales.

The vastness of Amazon’s catalog is what prompted us to publish this data in the first place. We want everyone to understand how our system works and why that means we sometimes miss price changes.

Amazon Price Checking - Product Prioritization

When checking product prices, we break products down into 9 groups based on locale — meaning the US products go in one group, UK in another, etc — and then break each of those groups down into 2 sub-groups. One (the “tracked” sub-group) contains all products which have at least one price watch, and the other (the “untracked” sub-group) contains the products with no price watches. All groups (and sub-groups) are updated in parallel, which usually means the smaller groups and sub-groups update faster than their counterparts. The lesson here is simple: if you want a product’s price checked regularly, create a price watch.

Now that you understand a little bit about how we queue products for price checking, on to the data!

Amazon Price Checking Statistics

As you can see, many of the products in our system are updated every few hours, some hourly, and some multiple times per hour. We’re working hard to improve the times of those big groups of products (US untracked, for example) and hope to bring our processing time for all Amazon products down to one day or less. Hourly updates for all Tracked products is also one of our goals.

Let us know what you think of this post in the comments. We could do more of this kind of thing if people are interested.

Amazon's Holiday Shipping Deadlines

camelytics — Sat, 17 Dec 2011 03:06:00 -0800

Just a quick reminder that you’re running out of days to place orders at Amazon if you want them to arrive by the 25th of December.

Read all about the US shipping deadlines here. In the UK? Here’s your link.

Good News for Our Amazon Users!

camelytics — Fri, 16 Dec 2011 22:00:00 -0800

Our users in Canada, Germany, Italy, Spain, and the United Kingdom are now able to access camelcamelcamel using localized domains which forward to the part of our site appropriate for your locale.

Here are the URLs:

And, to reduce the number of keystrokes needed to get to our site, all users can use 3camels.com to access our homepage; stick any locale at the beginning of that domain (ie, jp.3camels.com) and you’ll be taken to that part of our site. We recently liberated this domain from an Amazon employee (go figure!)

We are also increasing our price checking frequency in every locale, so expect to see prices change a lot more often in 2012 than they did in 2011! It’s going to be another great year at the Camel Farm.

Amazon's Lightning Deals

camelytics — Mon, 12 Dec 2011 14:47:38 -0800

Because we get our product data from Amazon’s API, and it does not provide prices for Lightning Deals, we are unable to show these prices on our site. In such cases, we receive and display the normal Amazon price.

Sorry for the inconvenience but this one is out of our hands.

We're Back Online!

camelytics — Fri, 09 Dec 2011 21:43:53 -0800

One of the main servers in our network decided to die tonight, so we were down for about an hour while we re-installed it. No data was lost, and we were still collecting data during that time, but no web requests were served.

Also, as of a few days ago, we are again able to collect Newegg data. It must have been a temporary bug in Newegg’s system.

Sorry for the inconvenience! I hope our upcoming site updates will make it up to you all.

Newegg Data is Unavailable

camelytics — Thu, 01 Dec 2011 12:49:00 -0800

We aren’t sure why but, within the last couple of days, our source of Newegg data suddenly disappeared. As we have been kicked out of their affiliate program, we don’t really have a good way to find out what’s going on. However, we have heard from one brave Camel user that Newegg’s site was down yesterday, so they might just be having temporary problems…

We will post again when we find out more, and are very sorry about the inconvenience.

Database Issue Resolved

camelytics — Sat, 26 Nov 2011 09:10:45 -0800

This morning, our database server filled up its storage and took our system down for about 10 minutes. No data was lost, but we did have to log everyone out to clear up some storage space. Everything is now back to normal and running smoothly.

Sadly, these “issue resolved” blog posts are becoming a trend. One we hope to break immediately!

Email Issues Resolved

camelytics — Mon, 21 Nov 2011 09:44:32 -0800

This morning we had a problem with our server that processes user emails which resulted in no emails being sent. This has been resolved, and all signup and alert emails will be delivered shortly.

Sorry about that! We hope to have these kinks worked out before Black Friday!

Sessions Problems Fixed!

camelytics — Wed, 09 Nov 2011 17:49:45 -0800

You’ll have to clear your cookies, but otherwise things should be back to normal as far as staying logged in across our various Amazon locales goes. Please comment on this post if you have cleared your cookies and are still being logged out when changing locales.

User Sessions are Acting Strange

camelytics — Tue, 08 Nov 2011 23:34:46 -0800

Before I go to bed, I want to let everyone know that there is a known bug in our site’s session system which has two side effects: 1) if you login in one locale, you won’t stay logged in when navigating to another locale; 2) login sessions expire much more rapidly than usual. Nothing major, just little annoyances.

Please keep this in mind when using the site in the next 12 or so hours, and I will do my best to fix these problems when I wake up!

Bug Fixes for our Amazon Locales

camelytics — Tue, 08 Nov 2011 12:33:16 -0800

We launched some wishlist synchronization bugfixes for the new Amazon locales (China, Italy, Spain.) We also fixed some product URL detection bugs in our search code.

Users in the new locales should now be able to sync their wishlists. And all users will (or should, anyway) appreciate our search engine correctly detecting and parsing product URLs. Hooray for working Camels!

A Summary of Recent Changes

camelytics — Sat, 05 Nov 2011 18:43:02 -0700

It has been a turbulent couple of weeks for us, but we’ve survived. Here’s what we’ve been up to.

Amazon API Changes

Think of Amazon’s API as our data source. They recently made some changes that effect what data is provided to users of the API. The net result for us is simple: if Amazon has the lowest price on a product, we cannot also get the Third Party New price; otherwise, we will still be able to get the Third Party New price, and used prices aren’t affected in either scenario. There is no published way to work around this, so it appears that we are stuck with this new API. Fortunately, most of our users probably won’t care about third party merchants when Amazon is selling for less, but we don’t like having holes in our data!

Bringing our code up-to-date with these API changes also means that we are now able to offer support for Amazons China, Italy, and Spain. Welcome to all of the new users who join us in those locales!

Top Drops Improvements

Due to the rapid fluctuation of prices at Amazon, our Top Drops list used to have the tendency to display products that weren’t actually worth looking at; the prices were still high, or they had jumped up and then right back down to the same price (or higher!)

These problems should now be a thing of the past. We’ve tightened the requirements for being displays in the Top Drops by addressing the aforementioned problems of bad prices and “fake” deals. We’re still tweaking the code as new products are updated and enter the list, but the list is already looking much more useful!

Ongoing Site Redesign

In parallel with those changes, we have been working on converting more of the site’s pages to our new style. In this update, the product browser’s default screen and the last page of the user registration process have had the new style applied.

We also fixed a bug in the wishlist management code that was causing some page requests to fail.

That’s it for now. Thanks to everyone who voted in our poll, you all had a lot of useful feedback!

The Cheapest Days of the Week to Shop Online

camelytics — Wed, 02 Nov 2011 18:39:55 -0700

The Cheapest Days of the Week to Shop Online

We're Back!

camelytics — Tue, 01 Nov 2011 23:12:50 -0700

Amazon API support is back online. This means that we are again collecting pricing data for all of our existing locales. Once we’re sure the code is stable, we’ll be launching support for Amazons China, Italy, and Spain.

We’ll post a full follow-up of the recent API changes and their effects on our site soon!

There will be No Pricing Updates this Morning

camelytics — Tue, 01 Nov 2011 03:45:57 -0700

Amazon officially launched the new version of their API today. Our code to support it is ready, but unfortunately I have to catch an early flight this morning.

Once I land in San Francisco and get home, I will update our servers and things should be back to normal. Until I get home and launch the code, there will be no product updates in any locale. I expect this to happen around 12pm Pacific time.

Sorry for any inconvenience or missed deals this may cause.

Please Vote in our Poll

camelytics — Sun, 30 Oct 2011 13:04:00 -0700

We really want to know what you think about Amazon’s API changes and, specifically, how they will change the functionality of our site and whether that will affect you.

Please vote, it only contains one question!

http://cosmicshovel.polldaddy.com/s/how-will-amazon-s-api-changes-affect-you

And here is some background on the aforementioned API changes, for those who haven’t read our previous blog post.

What Amazon's API Changes Mean for Us

camelytics — Fri, 28 Oct 2011 14:58:00 -0700

Amazon has again modified the product pricing data they share with us. We assumed that, although this was going to take place on October 25th, Amazon’s statement that these changes would not be mandatory until February 2012 meant we could continue along as usual and try to find a workaround before February. Wrong! It turns out that Amazon has retroactively applied these changes to all existing versions of their API (data source), and that means we’ve had to take some measures to protect our database from this “lesser” data while we figure out how to deal with it.

So, for now — and we really hope this is temporary — Amazon Germany and Amazon Japan support have been disabled. You can still view our existing data, but no new data will be consumed until we find a workaround. All other Amazon locales will continue working normally, with one caveat: due to heightened traffic, no new products will be added to our database until we can expand our updating capacity.

We are working with Amazon to resolve all of these issues, but it might take some time before we are back to running at full steam. We are sincerely sorry that this happened, but that’s how it goes when you rely on another company for data: you take what you can get and roll with the punches. And, due to our immense respect for Amazon, we want to avoid screen scraping if at all possible, so using their API is the only way for us to acquire product data.

Thanks for understanding.

Sign-ups are Working Again

camelytics — Fri, 28 Oct 2011 09:58:00 -0700

Last night around 10pm Pacific time, something went awry with our email system and caused all new user registrations to fail. We’ve spent the last couple of hours working around this problem, and believe we’ve got it fixed.

So, to anyone who tried and was unable to register in the last 13 or so hours, we apologize for the inconvenience and ask that you please give us another chance. We can change, honest!

Update

Our ISP has just confirmed that the email server problem was caused by one of their employees. ”Oops”.