Amazon or Netflix: Which is the better deal for subscribers?

Netflix stock took a tumble this week amid weak guidance and the announcement that Amazon is offering a new stand-alone streaming service for a dollar cheaper than its rival.

That dollar could be a big deal, but it doesn't say much about value. What exactly does a subscriber get for that money? Does one service have more shows or better content than the other?

Those are hard questions to answer, partly because the streaming services themselves make it hard for subscribers to know what exactly they're getting for their money. CNBC pulled data available on instantwatcher.com, a site that has access to both providers' databases, to estimate the value of each subscription.

At first glance, Amazon (AMZN) appears to have the upper hand. The data show nearly three times as many titles for Amazon than for Netflix (NFLX), and more of those titles are in the four- and five-star range according to each platform's five-point rating system.

But there are a few problems with taking those numbers at face value. First, not all those videos are the same length, and a two-hour long movie is arguably more valuable to a customer than a two-minute long clip.

It turns out that the average Netflix movie is actually 30 percent longer than the average Amazon offering. In fact, nearly half of the library's 18,000 titles that have run times have run times of an hour or less — in other words, not real movies at all. Only about 11 percent of Netflix movies with run times are that short.

More than 900 of Amazon's videos are less than 10 minutes long. Fewer than 30 Netflix videos have that distinction. Some of those videos, like the brief Amazon math videos "Problem Solving: Using Coordinates" or "Three Digit Numbers," don't seem terribly popular.

With the run-time data and a few assumptions, we can come up with a rough estimate how many hours of content each service offers. Surprisingly, the two platforms came out almost exactly the same in that calculation — about 36,000 hours worth of videos and television shows each.

Both Netflix and Amazon said that the reason they don't share the amount of streaming content that comes with a subscription is that the offerings change too frequently. Neither company commented on how close the CNBC.com estimates are to their own numbers.

A Netflix representative denied that the company even knows figures like the total number of hours or number of titles available on a given day, and declined to provide a list of titles available at this time.

"To put it simply, an exact number reflecting the breadth of the catalog does not exist," she said in an email. "The licensing and renewal process occurs on an on-going basis meaning titles are constantly being added and rotated off the service."

Both companies noted that they focus on quality rather than quantity. One way to measure the perceived quality of the content available (given that viewership metrics are also generally hidden from investors and the public) is to look at the star ratings actual users give each movie and television season. Netflix has a similar five-star system (although it may ditch it soon).

Overall, the average score for movies and television seasons is nearly the same for Netflix and Amazon — about 3.6 stars. Looking at just hours of content that are rated three stars and above, Netflix comes out ahead overall, despite Amazon's high-ranking titles.

Unfortunately, there are a few major caveats with this approach as well. Amazon's system records the number of people who have rated a movie, and a lot of those four- and five-star movies have been watched and rated by only a few people. As Hoony Youn, co-founder of the company that operates instantwatcher.com, put it: "If only five people have rated it, do you really trust that it's a five-star movie?"

Almost half of those five-star Amazon videos have been rated fewer than four times. That's how videos like "Using Coordinates" get a 5-star rating — from a single viewer. Here's what that five-star video looks like:

While Amazon shares the number of ratings a video has received (and instantwatcher.com can sort by that number), it isn't possible to weed out those possibly inaccurate ratings from the Netflix data.

It's also true that Netflix customers could just be pickier about movies than Amazon customers. Comparing about 350 videos that show up in both, the median for Amazon is about half a star higher. So perhaps it is appropriate to shift Amazon's ratings down some.

In the absence of any information from the companies themselves, it's hard to draw any ironclad conclusions, but it does appear that the two companies are neck and neck in the type of content they're offering. That means that a dollar difference in price could make a big difference, and other perks — like the ability to download on Amazon or the extra original content on Netflix — could sway customers who aren't sure which service to use.

Why is it so difficult to find out what comes with a streaming subscription? For many years, Netflix had a public API (a programming interface) that developers could use to pull information about its services. It shut down that access in late 2014, killing a number of applications with it.

After that, only a select group of websites, including instantwatcher.com, were kept alive with private licenses. But the ability to use the information from Netflix and other service providers often comes with strict rules about how it can be used, according to developers from some of those sites.

"Over the last five years, the categories have become far more complex and the players have started to put up more and more walls," said Nikhil Vaish, cofounder of CanIStream.it, one of the sites that stayed active after the initial public offering closed. "Rather than making it easier to find this stuff they've all made it harder."

Closed platforms are now the norm. That may be because having open access makes it easier for outside groups to do unflattering comparisons, like this AllFlicks analysis that found that the Netflix catalog was shrinking (AllFlicks was one of the partners that continued to function after the API went dark). Or it could be that the internet at large is simply moving away from the openness of APIs.

More From CNBC

• Top News and Analysis
  

• Latest News Video
  

• Personal Finance