What is machine learning and how does Netflix use it for its recommendation engine?

What is machine learning and how does Netflix use it for its recommendation engine?

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

What is machine learning and how does Netflix use it for its recommendation engine?

What is an online recommendation engine?

Think about examples of machine learning you may have encountered in the past such as a website like Netflix  that recommends what video you may be interested in watching next?
Are the recommendations ever wrong or unfair? We will  give an example and explain how this could be addressed.

2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams
2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams

Machine learning is a field of artificial intelligence that Netflix uses to create its recommendation algorithm. The goal of machine learning is to teach computers to learn from data and make predictions based on that data. To do this, Netflix employs Machine Learning Engineers, Data Scientists, and software developers to design and build algorithms that can automatically improve over time. The Netflix recommendations engine is just one example of how machine learning can be used to improve the user experience. By understanding what users watch and why, the recommendations engine can provide tailored suggestions that help users find new shows and movies to enjoy. Machine learning is also used for other Netflix features, such as predicting which shows a user might be interested in watching next, or detecting inappropriate content. In a world where data is becoming increasingly important, machine learning will continue to play a vital role in helping Netflix deliver a great experience to its users.

What is machine learning and how does Netflix use it for its recommendation engine?
What is machine learning and how does Netflix use it for its recommendation engine?

Netflix’s recommendation engine is one of the company’s most valuable assets. By using machine learning, Netflix is able to constantly improve its recommendations for each individual user.

Machine learning engineers, data scientists, and developers work together to build and improve the recommendation engine.

  • They start by collecting data on what users watch and how they interact with the Netflix interface.
  • This data is then used to train machine learning models.
  • The models are constantly being tweaked and improved by the team of engineers.
  • The goal is to make sure that each user sees recommendations that are highly relevant to their interests.

Thanks to the work of the team, Netflix’s recommendation engine is constantly getting better at understanding each individual user.

How Does It Work?

In short, Netflix’s recommendation algorithm looks at what you’ve watched in the past and then makes recommendations based on that data. But of course, it’s a bit more complicated than that. The algorithm also looks at data from other users with similar watching habits to yours. This allows Netflix to give you more tailored recommendations.

For example, say you’re a big fan of Friends (who isn’t?). The algorithm knows that a lot of Friends fans also like shows like Cheers, Seinfeld, and The Office. So, if you’re ever feeling nostalgic and in the mood for a sitcom marathon, Netflix will be there to help you out.

But That’s Not All…

Not only does the algorithm take into account what you’ve watched in the past, but it also looks at what you’re currently watching. For example, let’s say you’re halfway through Season 2 of Breaking Bad and you decide to take a break for a few days. When you come back and finish Season 2, the algorithm knows that you’re now interested in similar shows like Dexter and The Wire. And voila! Those shows will now be recommended to you.

Of course, the algorithm isn’t perfect. There are always going to be times when it recommends a show or movie that just doesn’t interest you. But hey, that’s why they have the “thumbs up/thumbs down” feature. Just give those shows the old thumbs down and never think about them again! Problem solved.

Another angle :

When it comes to TV and movie recommendations, there are two main types of data that are being collected and analyzed:

1) demographic data

2) viewing data.

Get 20% off Google Google Workspace (Google Meet) Standard Plan with  the following codes: 96DRHDRA9J7GTN6
Get 20% off Google Workspace (Google Meet)  Business Plan (AMERICAS) with  the following codes:  C37HCAQRVR7JTFK Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)

Active Anti-Aging Eye Gel, Reduces Dark Circles, Puffy Eyes, Crow's Feet and Fine Lines & Wrinkles, Packed with Hyaluronic Acid & Age Defying Botanicals

Demographic data is information like your age, gender, location, etc. This data is generally used to group people with similar interests together so that they can be served more targeted recommendations. For example, if you’re a 25-year-old female living in Los Angeles, you might be grouped together with other 25-year-old females living in Los Angeles who have similar viewing habits as you.

Viewing data is exactly what it sounds like—it’s information on what TV shows and movies you’ve watched in the past. This data is used to identify patterns in your viewing habits so that the algorithm can make better recommendations on what you might want to watch next. For example, if you’ve watched a lot of romantic comedies in the past, the algorithm might recommend other romantic comedies that you might like based on those patterns.

Are the Recommendations Ever Wrong or Unfair?
Yes and no. The fact of the matter is that no algorithm is perfect—there will always be some error involved. However, these errors are usually minor and don’t have a major impact on our lives. In fact, we often don’t even notice them!


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

The bigger issue with machine learning isn’t inaccuracy; it’s bias. Because algorithms are designed by humans, they often contain human biases that can seep into the recommendations they make. For example, a recent study found that Amazon’s algorithms were biased against women authors because the majority of book purchases on the site were made by men. As a result, Amazon’s algorithms were more likely to recommend books written by men over books written by women—regardless of quality or popularity.

These sorts of biases can have major impacts on our lives because they can dictate what we see and don’t see online. If we’re only seeing content that reflects our own biases back at us, we’re not getting a well-rounded view of the world—and that can have serious implications for both our personal lives and society as a whole.

One of the benefits of machine learning is that it can help us make better decisions. For example, if you’re trying to decide what movie to watch on Netflix, the site will use your past viewing history to recommend movies that you might like. This is possible because machine learning algorithms are able to identify patterns in data.

Another benefit of machine learning is that it can help us automate tasks. For example, if you’re a cashier and have to scan the barcodes of the items someone is buying, a machine learning algorithm can be used to automatically scan the barcodes and calculate the total cost of the purchase. This can save time and increase efficiency.

The Consequences of Machine Learning

While machine learning can be beneficial, there are also some potential consequences that should be considered. One consequence is that machine learning algorithms can perpetuate bias. For example, if you’re using a machine learning algorithm to recommend movies to people on Netflix, the algorithm might only recommend movies that are similar to ones that people have already watched. This could lead to people only watching movies that confirm their existing beliefs instead of challenged them.

Another consequence of machine learning is that it can be difficult to understand how the algorithms work. This is because the algorithms are usually created by trained experts and then fine-tuned through trial and error. As a result, regular people often don’t know how or why certain decisions are being made by machines. This lack of transparency can lead to mistrust and frustration.

What is Problem Formulation in Machine Learning and Top 4 examples of Problem Formulation in Machine Learning?

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

What are some good datasets for Data Science and Machine Learning?

This scene in the Black Panther trailer, is it T’Challa’s funeral?

r/marvelstudios - This scene in the Black Panther trailer, is it T’Challa’s funeral?

Recommended New Netflix  Movies 2022

World’s Top 10 Youtube channels in 2022

r/dataisbeautiful - [OC] World's Top 10 Youtube Channels of 2022

T-Series, Cocomelon, Set India, PewDiePie, MrBeast, Kids Diana Show, Like Nastya, WWE, Zee Music Company, Vlad and Niki

Techies and Geek Inspired Movies and TV Shows – Netflix Amazon Prime Video HBO YouTube TV

Techies and Geek Inspired Movies and TV Shows

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Techies and Geek Inspired Movies and TV Shows – Netflix Amazon Prime Video HBO YouTube TV

If you’re a movie buff or a geek, there’s no doubt you love spending your free time watching films and TV shows that inspire your passions. And in the age of Netflix, Amazon Prime Video, and HBO, there’s no shortage of inspiring content to watch. This blog post takes a look at some of the best geeky and tech-inspired movies and TV shows available on streaming services today. So whether you’re a die-hard Star Wars fan or you can’t get enough of Andy Griffith reruns, there’s something for everyone in this roundup!

As a software engineer who works long hours and go to kids activities after work, TV shows and movies help me relax after work.

This blog is an aggregate of trailers, questions and answers about of Geek inspired Movies and TV Shows.

Marvel Pictures! Here’s options for downloading or watching Black Panther 2: Wakanda Forever streaming the full movie online for free on 123movies & Reddit including where to watch Universal Pictures’ movie at home. Is Black Panther 2: Wakanda Forever 2022 available to stream? Is watching Black Panther 2: Wakanda Forever on Disney Plus, HBO Max, Netflix or Amazon Prime? Yes we have found an authentic streaming option / service. Details on how you can watch Black Panther 2: Wakanda Forever for free throughout the year are described below.

Watch Here: Black Panther 2: Wakanda Forever Free Streaming

Get 20% off Google Google Workspace (Google Meet) Standard Plan with  the following codes: 96DRHDRA9J7GTN6
Get 20% off Google Workspace (Google Meet)  Business Plan (AMERICAS) with  the following codes:  C37HCAQRVR7JTFK Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)

Active Anti-Aging Eye Gel, Reduces Dark Circles, Puffy Eyes, Crow's Feet and Fine Lines & Wrinkles, Packed with Hyaluronic Acid & Age Defying Botanicals

Is Black Panther 2: Wakanda Forever on Netflix? Black Panther 2: Wakanda Forever is not available to watch on Netflix. If you’re interested in other movies and shows, one can access the vast library of titles within Netflix under various subscription costs depending on the plan you choose: $9.99 per month for the basic plan, $15.99 monthly for the standard plan, and $19.99 a month for the premium plan. Is Black Panther 2: Wakanda Forever on Hulu? They’re not on Hulu, either! But prices for this streaming service currently start at $6.99 per month, or $69.99 for the whole year.

Hulu + Live TV. Is Black Panther 2: Wakanda Forever on Disney Plus? No sign of Black Panther 2: Wakanda Forever on Disney+,which is proof that the House of Mouse doesn’t have its hands on

Watch Here: Black Panther 2: Wakanda Forever Free Streaming


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

every franchise! Home to the likes of ‘Star Wars’, ‘Marvel’, ‘Pixar’, National Geographic’, ESPN,

STAR and so much more, Disney+ is available at the annual membership fee of $79.99, or the

monthly cost of$7.99. If you’re a fan of even one of these brands, then signing up to Disney+ is definitely worth it, and there aren’t any ads, either.

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

Is Black Panther 2: Wakanda Forever on HBO

Max? Sorry, Black Panther 2: Wakanda Forever is not available on HBO Max. There is a lot of

content from HBO Max for $14.99 a month, such a subscription is ad-free and it allows you to access all the titles in the library of HBOMax. The streaming platform announced an ad-supported version that costs a lot less at the price of $9.99 per month.

Is Black Panther 2: Wakanda Forever on Amazon

Video? Unfortunately, Black Panther 2: Wakanda Forever isnot available to stream for free on

Amazon Prime Video. However, you can choose othershows and movies to watch from there as it has a wide variety of shows and movies that youcan choose from for $14.99 a month. Is Black Panther 2:

Wakanda Forever on Peacock?

Black Panther 2: Wakanda Forever is not available to watch onLine

Peacock at the time of writing. Peacock offers a subscription costing $4.99 a month or $49.99 per year for a premium account. As their namesake, the streaming platform is free with content out in the open, however, limited.

Is Black Panther 2: Wakanda Forever on Paramount Plus?

Djamgatech: Build the skills that’ll drive your career into six figures: Get Djamgatech.

Black Panther 2: Wakanda Forever is not on Paramount Plus. Paramount Plus has two subscription options: the basic version ad-supported Paramount+ Essential service costs$4.99 per month,

How long have you fallen asleep during Black Panther (2018) Movie? The music, the story, and the message are phenomenal in Black Panther. I have never been able to see another Movie five times like I did this. Come back and look for the second time and pay attention.

Watch Black Panther (2018) WEB-DL movies This is losing less lame files from streaming Black Panther (2018), like Netflix, Amazon Video. Hulu, Black Panther (2018) chy roll, Discovery GO, BBC iPlayer, etc. These are also movies or TV shows that are downloaded through online distribution sites, such as iTunes.

Is Black Panther 2: Wakanda Forever on Amazon Prime?

Amazon Prime is not streaming Black Panther 2: Wakanda Forever movies. However, the streamer has a wide range of latest movie collections for their viewers, including Train to Busan, The Raid: Redemption, Hell or High Water, The Florida Project, and Burning.

Is Black Panther 2: Wakanda Forever on HBO Max?

No. Black Panther 2: Wakanda Forever is a Sony movie, not a Warner Bros. movie. Also, HBO Max will no longer be streaming theatrical movies in 2022. (Last year, Warner Bros. opted to simultaneously release its theatrical slate on streaming, meaning HBO Max subscribers could watch movies like Matrix Resurrections at home. This year, however, Warner Bros. theatrical movies will have a 45-day theaters-only run before moving to HBO Max.)

Techies and Geek Inspired Movies and TV Shows  - Netflix Amazon Prime Video HBO YouTube TV

Lupita Nyongo will be the new Black Panther. The opening of the trailer shows her in the Black Panther suit and she is missing from the rest of the trailer.

n the trailer the Queen/Bassett says her entire family is dead. It also shows Lupita giving birth to T’Challa’s child.

Ace the Microsoft Azure Fundamentals AZ-900 Certification Exam: Pass the Azure Fundamentals Exam with Ease

So, the Atlanteans attack. Shuri is killed in her attempt to take the BP mantle and defend Wakandan, or maybe captured and presumed dead. Then Nakia gives birth. Somehow a heart shaped herb is found and given to her, she is powered up and quickly recovers from giving birth (she may even take it while pregnant because she is seriously injured in the Atlantean attack.) Nakia becomes BP and queen regent. She fights Namor, and maybe also rescues Shuri.

Something like that.

 

Day Shift | Jamie Foxx, Dave Franco, and Snoop Dogg | Official Trailer | Netflix

source: r/movies

Day Shift:

Bud Jablonski (Jamie Fox) is a pool cleaner who scores extra cash as an exterminator, ridding the San Fernando Valley of its enduring vampire problem. But he does this work freelance—not part of the slayers union—so it’s not as lucrative as it is for his friend Big John Elliott (Snoop Dogg,) who is using his clout to bring Bud back to guild wages and benefits.

DEATH OF A LADIES’ MAN (2022) – Official Trailer

Trying to wrap my head around why they called the movie Death of a Ladies Man but used Bird on a Wire instead.

The movie was “inspired by the songs of Leonard Cohen.” So, I assume they’re going to use more than one.

Source: /r/movies

Ticket to Paradise | Official Trailer [HD]

What if in the end of Ocean’s Thirteen Danny and Tess got married, had a kid and divorced and now 20 years later try to sabotage their child’s engagement.

Looks like a rom com where there’s not a single unpredictable moment and yet will still be kind of fun.

Bodies Bodies Bodies | Official Trailer 2 HD | A24

SEE HOW THEY RUN | Official Trailer | Searchlight Pictures

It’s obvious from the popularity of “only murders in the building”, “knives out”, and the recent Christie movies that we’re entering a new era of campy mystery movies and I couldn’t be more excited for it.

source: r/movies

Thirteen Lives – Official Trailer | Prime Video

Just watching the trailer is making me feel claustrophobic.

source: r/movies

NOPE – Official Alternate Trailer – July 22

s it just me or do all of the night scenes look like they’re filmed in the day and color shifted to look like it’s night?

I remember they did that in 28 weeks later and it looked really jarring and cheap.

source: r/movies

Honor Society | Official Trailer | Paramount+ | July 29th

It wrinkles my brain that McLovin is now old enough to play a teacher.

source: r/movies

Vikrant Rona | Official trailer (English) | releasing on July 28

Kannada film industry producing really great movies.., visuals are at Hollywood level.., KGF 2, 777 Charli, Vikrant Rona..

What TV show was amazing at first but became unwatchable for you later on?

Heroes

Season 1 was great and fresh. Season 2 didn’t know what to do with itself and just started giving everyone super powers.

By Season 3, characters were just changing motivations at the drop of a hat and it was just a huge mess of bad writing.


The blacklist, so many loopholes and a never ending plot. I mean, the female hero (forgot her name) was wanted and had her pictures broadcast nationwide live, but a couple of weeks after she can do undercover work.


Not the worst offender, but That 70’s Show tanked pretty hard once Eric left. He was sorely needed to make the chemistry of the group work.


Once Upon a Time. The first 3 seasons were good! And then after that they just kept getting worse


source: r/askreddit

 

What are movies where the final line of dialogue is the title of the movie?

I was watching Chinatown last night (great movie, btw!) and got to the final scene already knowing what the last line is, cause it’s famous and all, but only then clicked that the last word spoken is the title of the movie. (I know you can hear some cops sorta talking in the background as the camera pans out but I don’t think anyone would say that is the final dialogue in the movie).

Anyway, got me thinking, what are some other movies like this? Last words are the title of the film.

1- “And then they realized they were no longer little girls…They were little women.” – As read by Moe

2- Perfection

3- “I am Iron Man

4- The Breakfast Club

5- “It’s Fantastic.”

6- “Say that again”

7- “It’s Fantastic”

8- “Guys, I think I’ve got it”

9- Fant4stic

10- I told my wife about my dinner with Andre

11- Queen’s Boulevard

12- “By the end of that summer of ’59, we truly were a Fellowship of the Ring.”

13- Mystery men ends with a reporter asking “who are these Mystery men?”

14- “Finally, an end to these Star Wars”

source: r/movies

 

I’d argue he’s the second most powerful member of The Seven

(Note, I’m doing TV Show Black Noir)

Black Noir is my favorite character in The Boys only second to Stormfront (cause she’s hot). So I always wanted to publicly share where I scale Noir, but before I do this, I have to share a few stuff stuff.

  1. The Boys Presents Diabolical is mostly canon, specifically episodes 6,7 and 8.

That’s really all, now sit back and enjoy (or dislike) this answer.


  • Section 1: AP

Black Noir’s AP feat-wise isn’t too impressive since his speed is the real deal, but he’s still no joke since he’s able to fight and defeat Kimiko in a battle.

 
 

(I consider this fight to be an outlier since Black Noir should be faster than her, but it doesn’t change the fight since Black Noir is stronger)

And keep in mind this is the same Kimiko that survived attacks from A-Train

 
 

And even reacted to A Train while he was going fast

 
 
Kimiko speed feat
Imgur: The magic of the Internet
 

This says a lot since she was badly damaged while fighting Noir (Had to use her self healing to survive), and she didn’t even get harmed that much while fighting A-Train since she got back up a few seconds later.

And he has shown himself to be much stronger than Starlight in multiple instances

 
 

https://giant.gfycat.com/ImpossibleExhaustedHarvestmouse.mp4

This is the same Starlight that survived an attack from Stormfront (Same with Kimiko)

 
 

And Starlight tanked a Serbu BFG 50A with no injury

 
 
Starlight durability #1
Imgur: The magic of the Internet
 

So already Noir scales above Kimiko and Starlight and scales above A Train due to a better performance aganist Kimiko, you could also argue his AP being at least on par with Maeve due to the tree nut scene and her deciding to use tree nuts instead of fighting him.

 
 

But it’s an iffy argument, doesn’t really matter though cause I think Noir beats her regardless.


  • Section 2: Speed

My favorite section yet.

Noir is stated to be much faster than a car

Not too impressive but this is just the start.

Black Noir was consistently able to dodge Homelander’s beams

And dodged Homelander himself

 
 
Blacknoirvshomelander GIF – Find & Share on GIPHY
Discover & share this Blacknoirvshomelander GIF with everyone you know. GIPHY is how you search, share, discover, and create GIFs.
 

I made an answer as to why Homelander is Hypersonic+ at least and his lasers are faster, check it out

 
 

Homelander is narratively potrayed to be the most powerful being in The Boys and should be the fastest

Not only that but Black Noir is consistently able to counter Homelander’s beams with his knives and is fast enough to throw them

https://i.giphy.com/media/mopdzvwJYP1FbseXKL/giphy.mp4 https://i.giphy.com/media/JrxshenT8yh7W7kpxi/giphy.mp4

No one in The Boys has shown a good speed feat like Black Noir just for this (excluding Homelander) not even A Train has shown anything like this.


  • Section 3: Durability

Doesn’t show too much but he’s able to tank a bomb from Naqib

 

With minor injuries to him

And tanked explosions previously as well

https://giant.gfycat.com/AdorableAbandonedInchworm.mp4 https://giant.gfycat.com/ElementarySpeedyDodo.mp4


  • Section 4: Others

Not too much but he’s great at Martial Arts, Stealth, Weapons, etc. He’s basically Batman

I hope you enjoyed the answer, wanna thank some people who remain annoymous for proving the scans to some of these

The Western TV shows that are usually considered the best, are the ones from the 50s and 60s.

Rawhide (1959 – 1965).

Bonanza (1959 – 1973).

Wagon Train (1957 – 1965).

Maverick (1957 – 1962).

Have Gun – Will Travel (1957 – 1963).

The Wild Wild West (1965 – 1969).

Gunsmoke (1955 – 1975).

Thanks for reading. Source: https://qr.ae/pvPYMs

The Gray Man | Official Trailer | Netflix

  • Is Indiana Jones and the Last Crusade the best adventure movie?
    by /u/jobbitonfan on April 16, 2024 at 1:08 pm

    This cinematic gem, released in 1989, is the third installment in the iconic Indiana Jones film series. Directed by the legendary Steven Spielberg and starring the incomparable Harrison Ford as the titular archaeologist, this adventure flick takes us on a thrilling quest for the Holy Grail. Is Indiana Jones and the Last Crusade the best adventure movie? - cineypalomitas submitted by /u/jobbitonfan [link] [comments]

  • Beatles’ 1970 ‘Let It Be’ Documentary, Out of Circulation for Four Decades, Headed to Disney+ on May 8 After Restoration by Peter Jackson’s Team - Official Poster
    by /u/MarvelsGrantMan136 on April 16, 2024 at 1:04 pm

    submitted by /u/MarvelsGrantMan136 [link] [comments]

  • I saw the devil.
    by /u/prettywilted on April 16, 2024 at 12:27 pm

    It was such a good movie (i had watched the other movie which was heavily inspired from it). I'm just so much into psychological thrillers so it was perfect but i was more irritated with the explicit scenes rather than the gore like ugh༎ຶ‿༎ຶ and since i liked it so much, looking for similar movie plots. Is there any?;) submitted by /u/prettywilted [link] [comments]

  • movies that made you go 'holy shit I love movies'
    by /u/No-Maintenance-3010 on April 16, 2024 at 11:45 am

    I've been watching a lot less movies lately, just haven't been feeling it so much, yknow? Anybody have any movies that when they ended and you were just like 'holy fucking shit movies I love movies!' Or like movies that were just so good you were like that was so fucking good I need to watch more movies asap? submitted by /u/No-Maintenance-3010 [link] [comments]

  • Movies/shows that you love, but you hate or can't stand the fandom
    by /u/phantom_avenger on April 16, 2024 at 11:43 am

    Whenever I get invested in a movie or show, I am very eager to start having a discussion with other people who have seen it where I'll go on Reddit and explore if there are any subreddits where users can have those discussions or view any memes people came up with that might make me laugh or hit deep depending on the context. But I was curious if there is any movie or show that people love to watch, but can't stand the fandom behind it because everytime you want to start a discussion, have an opinion or are quite critical towards a certain character. People get insecure and weird and start throwing uncalled for insults and attacks on you, especially if it involves "character shipping wars". One example for me with a show is Netflix's Sex Education, I really enjoyed this show up until the last season where the quality dropped. But the way fans on there get so toxic about Otis, Maeve and Ruby is just wild and exhausting! I speak as someone who did prefer Otis and Ruby together, and while I understand people who ship the other couple (Otis and Maeve) I just don't enjoy it as much as they do and I get hit by a stampede of those fans that are so protective of it they'll attack anyone that has an opinion that differs from their own. submitted by /u/phantom_avenger [link] [comments]

  • Name the last movie you have watched that was so good you had to rewatch it multiple times
    by /u/DogsFloppyDisk on April 16, 2024 at 11:17 am

    Some movies are so good you want to watch them again and again. However, I find these movies are rare. Please name a movie that was released in the last 10 years that left a deep mark. The genre doesn't matter. Just movies that captivated you because of the story, imagery, acting...and left you with that "WOW" feeling. submitted by /u/DogsFloppyDisk [link] [comments]

  • How many times in movies, where a character says the name the name of a famous celebrity. Only for that same celebrity to join the film later on?
    by /u/ah-screw-it on April 16, 2024 at 8:59 am

    (I know its hard to describe, but let me explain) In the first sonic movie, there's a scene where Sonic is watching speed with Keanu Reeves, and mentions him by name. But now that Keanu has been cast as Shadow for sonic 3. That leaves the speed scene in a bit of a weird place. And I wonder if there are scenes like that where an actor is sort of retconned into a roll. (And I'm not talking about it being explained by multiverse crap like in endgame or no way home) submitted by /u/ah-screw-it [link] [comments]

  • Are there any groups of movies you can't believe were directed the same person?
    by /u/NolanLover on April 16, 2024 at 8:44 am

    For example I can't believe the 2000 Grinch movie and the Han Solo movie were directed by Ron Howard. The same person who directed the Robert Langdon movies and Apollo 13 I can't believe Elf, Zathura and 2019's The Lion King were directed by Jon Favreau. The same person who directed the 1st 2 Iron Man movies I can't believe Percy Jackson 1 and Pixels were directed by Chris Columbus. The same person who directed the 1st 2 Harry Potter movies and the 1st 2 Home Alone movies I can't believe Forrest Gump and Cast Away were directed by Robert Zemeckis. The same person who directed the Back to the Future movies, Roger Rabbit and The Polar Express I have more examples but in some of the cases are movies I never actually watched submitted by /u/NolanLover [link] [comments]

  • Poster for ‘You Can't Run Forever’, starring J.K. Simmons
    by /u/JonasKahnwald11 on April 16, 2024 at 8:15 am

    submitted by /u/JonasKahnwald11 [link] [comments]

  • There was no reason that Hugo Weaving couldn’t have played Smith in Matrix Resurrections
    by /u/HotlineBirdman on April 16, 2024 at 3:07 am

    This isn’t a knock against Jonathan Groff and it isn’t a laundry list of complaints about The Matrix Resurrections, but… nothing in this film actually specifically needed Smith to be recast. Okay, Morpheus could be hand waved as a younger version and technically Neo created him from scratch, but Smith… is just Smith, shoved back in the Matrix with his mind wiped. Why couldn’t Hugo Weaving played Smith as Thomas Anderson’s friend and business partner/contemporary. Let that relationship play out with actual warmth and camaraderie until the truth is revealed and you can see the old Smith reawaken and shed this friendly persona and become this petty vengeful madman. Given the weight of the conflict between Neo and Smith in the previous films, I think this would have been a better casting choice keeping all other elements of the film the same. submitted by /u/HotlineBirdman [link] [comments]

  • What’s a movie you can’t watch anymore?
    by /u/seveer37 on April 16, 2024 at 12:57 am

    I know this may be a weird question to ask but I tried rewatching the Parent Trap a while back. I used to love this movie as a kid! It was so fun, and happy. But now as an adult I just couldn’t do it. Life is nowhere near like this. Divorced couples hardly ever get back together, families are never this happy, and love doesn’t solve all the problems that pop in life. submitted by /u/seveer37 [link] [comments]

  • What are some funny or strange quotes from the movies that you enjoy repeating?
    by /u/fink_barton on April 16, 2024 at 12:40 am

    Some movie quotes make little sense when someone is not aware of the context. For example, sometimes when a person says no to a couple of requests, I say: "A schmoke and a pancake?" When they pause or say what, I then add, "You know, a flapjack and a schigarette? No? Schigar and a waffle? No?....Well, then there ish no pleashing you." This is my attempt to imitate Goldmember: https://youtu.be/_LlPU6KenjU?si=rSMoKqzfZPR-mepz&t=12 Any funny or odd lines from the movies you enjoy repeating? submitted by /u/fink_barton [link] [comments]

  • Who is a one-hit wonder filmmaker?
    by /u/devpuppy on April 16, 2024 at 12:23 am

    It seems like it’s rare in the film world for a director to do one buzzy movie and then disappear. Most filmmakers keep doing new projects (even if they don’t land) and fizzle out rather than disappear. Has there ever been an auteur who released one iconic movie, even an indie or cult movie, and then left everyone waiting for more forever? The movie equivalent of the recording artist Gotye: a one-hit wonder. submitted by /u/devpuppy [link] [comments]

  • There's a weird movie that my mom can't remenber the name, only a scene.
    by /u/HourTap6580 on April 15, 2024 at 10:45 pm

    So, some time ago, my mom said she has a memory of a movie scene, but dont remenber where it was. basically, it was a scene where a mother drowns her own kid, because the kid is possesed by a demon. And it was probably a horror movie. Can someone try to find which movie it is? If yes, thank you. edit: i forgot to say smtg. The mother was drowning the kid in a bathtub submitted by /u/HourTap6580 [link] [comments]

  • When was the last time there was a genuine “I didn’t see that coming” moment in a big blockbuster movie? Not because you personally avoided the spoiler but because it was never leaked.
    by /u/Deserana12 on April 15, 2024 at 9:54 pm

    Please for the love of Christ note the “big blockbuster movie” because thats the point of this thread, we’re all aware Sorry to Bother You takes a turn! But someone mentioned in the Keanu Sonic thread about how it’s possible it was leaked when the real reveal may have supposed to have been when Knuckles debuts next week. And if so, that’s a huge shame and a huge issue I have with modern movies. Now I know that’s not the biggest thing ever but it did make me think about how prevalent spoilers are in the movie sphere and how much it has tainted movies, to the point some Redditors can’t probably imagine what it would have been like watching something like The Matrix, The Empire Strikes Back or even something like Cloverfield for the first time in a theater. Massive movies with big reveals designed to not be revealed until opening night. Even with things like Avengers Endgame, it was pretty well known that Iron Man would die. I think Interstellar after Cooper goes into the black hole was the last time I genuinely had no idea what was going to happen because as far as I remember no marketing spoiled it and there weren’t any super advanced leaks other than original script which wasn’t the final version. So I’m just wondering what people would cite as the last big movie reveal in a huge blockbuster? submitted by /u/Deserana12 [link] [comments]

  • Keanu Reeves Joins ‘Sonic 3’ as Shadow
    by /u/MarvelsGrantMan136 on April 15, 2024 at 8:07 pm

    submitted by /u/MarvelsGrantMan136 [link] [comments]

  • The Big Short, Too Big to Fail and Margin Call don’t get enough credit for collectively being 3 great movies that deal with 2008 financial crisis.
    by /u/SerDire on April 15, 2024 at 8:02 pm

    On their own, they are great stand alone movies but when they are grouped together, it becomes clear how great they truly are because they each deal with a unique specific section of the 2008 financial crisis. The Big Short is probably the most famous one and it mostly deals with investors trying to see ahead and how they can potentially get rich by betting against the housing market. You get to see the POV of the top investors who are trying to capitalize on everyone else’s pain and misery. It’s serious stuff but this is easily the most light hearted and funny one of the three. Too Big to Fail is a close look at the government’s response to the financial crisis. You have the secretary of the treasury, the chair of the federal reserve and countless other government officials trying to save the system. They also bring in the top guys from all the major banks to stop the bleeding. This is a look from inside the government’s POV Margin Call takes place over one day and it deals with a single business trying to see what is happening, why it’s happening and how bad it’s going to get. Jeremy Irons plays the CEO and he doesn’t not give one shit about how his actions will hurt the system, he just wants his business to survive. Grouped together, all three give you a somewhat cohesive look at how various sections of the financial system looked and handled the 2008 financial crisis. submitted by /u/SerDire [link] [comments]

  • Brooke Shields' biggest films were popular and shocking. How were they able to get away with it?
    by /u/Azul_Lawson on April 15, 2024 at 6:42 pm

    I didn't know she was a minor when she did Endless Love and The Blue Lagoon. Thankfully, they used a body double for the racier scenes in those films, yet I can't watch those films and not be uncomfortable. Especially with "Endless Love" which is a erotic film. "The Blue Lagoon" is surprisingly innocent. If anyone is objectified in "The Blue Lagoon", it is Christopher Atkins who was over 18 and did a lot of ass shots throughout the film. Brooke Shields has a complex relationship with those films. She used to be proud and I remember she talked about Endless Love like it was her biggest achievement as an actress. After she did her documentary, she appeared to disown these films. submitted by /u/Azul_Lawson [link] [comments]

  • ‘Rust’ Armorer Hannah Gutierrez-Reed Sentenced to 18 Month Prison Term For Involuntary Manslaughter
    by /u/MarvelsGrantMan136 on April 15, 2024 at 6:16 pm

    submitted by /u/MarvelsGrantMan136 [link] [comments]

  • HBO Max has a problem with starting the picture-in-picture recommendations screen too early and Only Yesterday (1991) is a huge victim of this
    by /u/YoungTroubadour on April 15, 2024 at 4:47 pm

    If you're not familiar, there's a beautiful final scene where the main character is on a train back to the city. As the credits start to appear on screen, she has a change of heart and returns to the countryside she was leaving, accompanied by memories of her childhood that she had been thinking back to during the film. Problem is, if you were watching this on Max you might've missed the entire resolution to the story because the streaming service cuts to the recommendations screen right as the first credits appear and you assumed the movie was done. I also noticed this with The Searchers (1956). Rather than holding on one of the most iconic final shots of all time for a few more seconds, the recommendations screen pops up right as Ethan turns to walk away. Are there any other examples of streaming services cutting away egregiously early? submitted by /u/YoungTroubadour [link] [comments]

  • What is the term for a character that the protagonists can understand but not the audience?
    by /u/InsideSpeed8785 on April 15, 2024 at 4:29 pm

    Is there an official film term for a character that talks in an indecipherable language to the audience yet the other characters in the movie seem to understand them? It’s speech, beeps, groans that are interpreted by the other characters and relayed to the audience. Examples include but are not limited to Chewbacca, R2D2, Cousin IT, Schnitzel from Chowder (TV but still), Groot. Who started that trend? submitted by /u/InsideSpeed8785 [link] [comments]

  • Netflix’s New Film Strategy: More About the Audience, Less About Auteurs
    by /u/mrnicegy26 on April 15, 2024 at 3:07 pm

    submitted by /u/mrnicegy26 [link] [comments]

  • What was the first movie you remember seeing in theaters? And I mean REMEMBER.
    by /u/AceCoordinatorMary on April 15, 2024 at 2:49 pm

    For me it's definitely Spider-Man 2002. I'm sure I've "seen" other films before that one (my mom likes to talk about how she and her friend took me and my sister to see the Lion King in theaters but I certainly don't remember that.), but this is an entire experience I remember to my core. I even remember the marketing beforehand; I will die on the hill that the Twin Tower setup was supposed to be Spider-Man's reveal in the film. 100%. But then you know...THINGS happened. The movie has of course not aged the best. But it will have a special spot in my heart for being one of the first movies I remember so vividly and its why Spider-Man is my favorite hero. He left an impact. I'm gonna make an edit and say that yes I did see Sorcerer's Stone in theaters, but I more so remember the experience of that night not the movie itself. I hope that makes sense. submitted by /u/AceCoordinatorMary [link] [comments]

  • Great actors whose last movie sucked
    by /u/ellaenchanted23 on April 15, 2024 at 10:53 am

    My dad and I were talking about Bela Lugosi, and how his last movie was Plan 9 from Outer Space, one of the most infamousbad movies of all time. Thats not to say he wasnt in other stinkers, but his once (stable) career really seemed to peter out at the end with a string of garbage. Another example is Joan Crawford, a true star, whose last movie was Trog, which Is also infamous for its camp and crappieness. Who else can you think of? submitted by /u/ellaenchanted23 [link] [comments]

  • Who has played an "old man" the longest?
    by /u/BusterBerg on April 15, 2024 at 10:27 am

    My dad once said something that has stuck with me for a long time. I was around 10 and I wondered how Sean Connery could have played so in so many films as a old man, and my father said: "Sean Connery has been old for a long time." I think Highlander was one of his first "wise old mentor"-roles, which was in 1986, and his last was League of Extraordinary Gentlemen in 2003. That's 17 years of playing "the old man". Has anybody done it for longer? submitted by /u/BusterBerg [link] [comments]

  • Official Discussion Megathread (Civil War / Don't Tell Mom the Babysitter's Dead / Arcadian / Sting)
    by /u/LiteraryBoner on April 12, 2024 at 9:39 pm

    Civil War Don't Tell Mom the Babysitter's Dead Arcadian Sting submitted by /u/LiteraryBoner [link] [comments]

  • Hi I’m Joshua Caldwell director/producer of MENDING THE LINE - AMA!
    by /u/Joshua_Caldwell on April 5, 2024 at 4:02 pm

    Hey Reddit! Director Joshua Caldwell here. My new film MENDING THE LINE, starring Brian Cox, Sinqua Walls, Perry Mattfeld, Wes Studi, and Patricia Heaton) was the #1 film on Netflix last week in the US and hit #10 on the Global Top Ten. Which is amazing for a little drama about PTSD and Fly Fishing! I’m excited to chat about MENDING THE LINE or anything else you’d like to know about me, my career, fly fishing, working with Brian Cox, etc. Ask me anything! Trailer: https://www.youtube.com/watch?v=tSkv_sEyPSc Mending the Line on Netflix: https://www.netflix.com/title/81768966 https://preview.redd.it/4rvj6x0kqosc1.jpg?width=2316&format=pjpg&auto=webp&s=6f2d53b25964ccda06fdd9a1ee2a7ff1832174ce submitted by /u/Joshua_Caldwell [link] [comments]


15 largest entertainment streaming companies in the world – Netflix leads the pack

Post image

80% of movies on Netflix were released after 2015 

80% of movies on Netflix were released after 2015 [OC]
80% of movies on Netflix were released after 2015

Data Sciences – Top 400 Open Datasets – Data Visualization – Data Analytics – Big Data – Data Lakes

What are some good datasets for Data Science and Machine Learning?

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Data Sciences – Top 400 Open Datasets – Data Visualization – Data Analytics – Big Data – Data Lakes

Data science is an interdisciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from structured and unstructured data, and apply knowledge and actionable insights from data across a broad range of application domains.

A dataset is a collection of data, usually presented in tabular form. Good datasets for Data Science and Machine Learning are typically those that are well-structured (easy to read and understand) and large enough to provide enough data points to train a model. The best datasets are often those that are open and freely available – such as the popular Iris dataset. However, there are also many commercial datasets available for purchase. In general, good datasets for Data Science and Machine Learning should be:

2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams
2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams
  • Well-structured
  • Large enough to provide enough data points
  • Open and freely available whenever possible

In this blog, we are going to provide popular open source and public data sets, data visualization, data analytics and data lakes.

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses
AWS Data Analytics Specialty Practice Exam 2023
AWS Data Analytics Specialty Practice Exams

Population Distribution of the World by Continent

Population Distribution of the World by Continent
Population Distribution of the World by Continent

 

Get 20% off Google Google Workspace (Google Meet) Standard Plan with  the following codes: 96DRHDRA9J7GTN6
Get 20% off Google Workspace (Google Meet)  Business Plan (AMERICAS) with  the following codes:  C37HCAQRVR7JTFK Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE (Email us for more codes)

Active Anti-Aging Eye Gel, Reduces Dark Circles, Puffy Eyes, Crow's Feet and Fine Lines & Wrinkles, Packed with Hyaluronic Acid & Age Defying Botanicals

Fertility rates all over the world are steadily declining

Yes, fertility rates have been declining globally in recent decades. There are several factors that contribute to this trend, including increased access to education and employment opportunities for women, improved access to family planning and birth control, and changes in societal attitudes towards having children. However, the rate of decline varies significantly by country and region, with some countries experiencing more dramatic declines than others.

 

The most Daily Wikipedia Page Views in 2022


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

How Americans Spend Their Money by Generation

How Americans Spend Their Money by Generation
How Americans Spend Their Money by Generation

 

AWS Data Analytics Specialty Practice Exam 2023
AWS Data Analytics Specialty Practice Exams

Largest countries in the world (by area size)

Largest countries in the world (by area size)
Largest countries in the world (by area size)

The Highest Grossing Movies Of All Time

The Highest Grossing Movies Of All Time

We are still living mostly on gas, oil & coal – Global primary energy consumption by source (TWh)

r/dataisbeautiful - [OC] We are still living mostly on gas, oil & coal - Global primary energy consumption by source (TWh)

Consumption vs production based CO2 emissions by country

Consumption vs production based CO2 emissions by country

Largest banks in the world by total assets

r/dataisbeautiful - [OC] Largest banks in the world by total assets

Inflation rate and nominal interest rate

Inflation rate and nominal interest rate
Inflation Rate 2022 – 2023 Inflation Reduction Act California Inflation Relief – Inflation Relief Checks

Police Killings per Capita v Homicide Rate per Capita for Select OECD Countries

r/dataisbeautiful - Police Killings per Capita v Homicide Rate per Capita for Select OECD Countries [OC]

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

Instagram Rich List 2022 – Which celebrity earns the most in the world in 2022?

Top human-caused threats to birds in the US

 Top human-caused threats to birds in the US
 Top human-caused threats to birds in the US

Trailblazing Scientists who Shaped our World

Trailblazing Scientists who Shaped our World
Trailblazing Scientists who Shaped our World

Suicide rate among countries with the highest Human Development Index

Percentage of Obesity in 2022

The United States is not the country with the highest obesity rates.
Percentage of Obesity in 2022

Source: https://worldpopulationreview.com/country-rankings/obesity-rates-by-country

Tools: Datawrapper

3 largest global payment networks – measured by total payment volume each year ($B)

3 largest global payment networks - measured by total payment volume each year ($B)

Stocks Vs Bonds 2022

Most expensive football transfers

Most expensive football transfers

11 developing countries with higher life expectancy than the United States

 
r/dataisbeautiful - [OC] 11 developing countries with higher life expectancy than the United States

Healthcare expenditure per capita vs life expectancy years

 
r/dataisbeautiful - [OC] Healthcare expenditure per capita vs life expectancy years

1.2% of adults own 47.8% of world’s wealth

r/dataisbeautiful - [OC] 1.2% of adults own 47.8% of world's wealth

How to Mathematically Win at Rock  Paper  Scissors

r/dataisbeautiful - [OC] How to Mathematically Win at Rock, Paper, Scissors

Researchers from IBM, MIT and Harvard Announced The Release Of DARPA “Common Sense AI” Dataset Along With Two Machine Learning Models At ICML 2021

Building machines that can make decisions based on common sense is no easy feat. A machine must be able to do more than merely find patterns in data; it also needs a way of interpreting the intentions and beliefs behind people’s choices.

At the 2021 International Conference on Machine Learning (ICML), Researchers from IBM, MIT, and Harvard University have come together to release a DARPA “Common Sense AI” dataset for benchmarking AI intuition. They are also releasing two machine learning models that represent different approaches to the problem that relies on testing techniques psychologists use to study infants’ behavior to accelerate the development of AI exhibiting common sense. 

Djamgatech: Build the skills that’ll drive your career into six figures: Get Djamgatech.

Source – Summary – Paper – IBM Blog

Largest healthcare services companies in the world

r/dataisbeautiful - [OC] Largest healthcare services companies in the world

Performance Of FAANG Stocks In 2022 (Apple, Amazon, Google, Meta, Netflix)

100 million protein structures Dataset by DeepMind

DeepMind creates ‘transformative’ map of human proteins drawn by AI. By the end of the year, DeepMind hopes to release predictions for 100 million protein structures, a dataset that will be “transformative for our understanding of how life works,

Here’s a good article about this topic

Google Dataset Search

Google Dataset Search

Malware traffic dataset

Comprises 1914081 records created from all malware traffic analysis .net PCAP files, from 2013 to 2021. The logs are generated using Suricata and Zeek.

Originator: ali_alwashali

2019 Crime statistics in the USA

Dataset with arrest in US by race and separate states. Download Excel here

Ace the Microsoft Azure Fundamentals AZ-900 Certification Exam: Pass the Azure Fundamentals Exam with Ease

Researchers from IBM, MIT and Harvard Announced The Release Of DARPA “Common Sense AI” Dataset Along With Two Machine Learning Models At ICML 2021

Building machines that can make decisions based on common sense is no easy feat. A machine must be able to do more than merely find patterns in data; it also needs a way of interpreting the intentions and beliefs behind people’s choices.

At the 2021 International Conference on Machine Learning (ICML), Researchers from IBM, MIT, and Harvard University have come together to release a DARPA “Common Sense AI” dataset for benchmarking AI intuition. They are also releasing two machine learning models that represent different approaches to the problem that relies on testing techniques psychologists use to study infants’ behavior to accelerate the development of AI exhibiting common sense. 

Source – Summary – Paper – IBM Blog

100 million protein structures Dataset by DeepMind

DeepMind creates ‘transformative’ map of human proteins drawn by AI. By the end of the year, DeepMind hopes to release predictions for 100 million protein structures, a dataset that will be “transformative for our understanding of how life works,

Here’s a good article about this topic

Google Dataset Search

Google Dataset Search

Malware traffic dataset

Comprises 1914081 records created from all malware traffic analysis .net PCAP files, from 2013 to 2021. The logs are generated using Suricata and Zeek.

Originator: ali_alwashali

 CPOST dataset on suicide attacks over four decades

The University of Chicago Project on Security and Threats presents the updated and expanded Database on Suicide Attacks (DSAT), which now links to Uppsala Conflict Data Program data on armed conflicts and includes a new dataset measuring the alliance and rivalry relationships among militant groups with connections to suicide attack groups. Access it here.

Credit Card Dataset – Survey of Consumer Finances (SCF) Combined Extract Data 1989-2019

 You can do a lot of aggregated analysis in a pretty straightforward way there.

Drone imagery with annotations for small object detection and tracking dataset

11 TB dataset of drone imagery with annotations for small object detection and tracking

Download and more information are available here

Dataset License: CDLA-Sharing-1.0

Helper scripts for accessing the dataset: DATASET.md

Dataset Exploration: Colab

NOAA High-Resolution Rapid Refresh (HRRR) Model

The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.

When will computers replace humans?

r/dataisbeautiful - [OC] When will computers replace humans?

This chart is essentially measuring “How good is a human at a computers’ area of strength”.. meanwhile computers simply can not compete in human areas of strength.

The most popular car brands on Reddit

The most popular car brands on Reddit
The most popular car brands on Reddit

Fruit Efficient-C Analysis

Fruit Efficient-C Analysis
Fruit Efficient-C Analysis

Registry of Open Data on AWS

This registry exists to help people discover and share datasets that are available via AWS resources. Learn more about sharing data on AWS.

See all usage examples for datasets listed in this registry.

See datasets from Digital Earth AfricaFacebook Data for GoodNASA Space Act AgreementNIH STRIDESNOAA Big Data ProgramSpace Telescope Science Institute, and Amazon Sustainability Data Initiative.

Textbook Question Answering (TQA)

1,076 textbook lessons, 26,260 questions, 6229 images

Documentation: allenai.org/data/tqa

Download

Harmonized Cancer Datasets: Genomic Data Commons Data Portal

The GDC Data Portal is a robust data-driven platform that allows cancer
researchers and bioinformaticians to search and download cancer data for analysis.

Genomic Data Commons Data Portal
Genomic Data Commons Data Portal

The Cancer Genome Atlas

The Cancer Genome Atlas (TCGA), a collaboration between the National Cancer Institute (NCI) and National Human Genome Research Institute (NHGRI), aims to generate comprehensive, multi-dimensional maps of the key genomic changes in major types and subtypes of cancer.

AWS CLI Access (No AWS account required)

aws s3 ls s3://tcga-2-open/ --no-sign-request

Therapeutically Applicable Research to Generate Effective Treatments (TARGET)

The Therapeutically Applicable Research to Generate Effective Treatments (TARGET) program applies a comprehensive genomic approach to determine molecular changes that drive childhood cancers. The goal of the program is to use data to guide the development of effective, less toxic therapies. TARGET is organized into a collaborative network of disease-specific project teams.  TARGET projects provide comprehensive molecular characterization to determine the genetic changes that drive the initiation and progression of childhood cancers. The dataset contains open Clinical Supplement, Biospecimen Supplement, RNA-Seq Gene Expression Quantification, miRNA-Seq Isoform Expression Quantification, miRNA-Seq miRNA Expression Quantification data from Genomic Data Commons (GDC), and open data from GDC Legacy Archive. Access it here.

Genome Aggregation Database (gnomAD)

The Genome Aggregation Database (gnomAD) is a resource developed by an international coalition of investigators that aggregates and harmonizes both exome and genome data from a wide range of large-scale human sequencing projects. The summary data provided here are released for the benefit of the wider scientific community without restriction on use. Downloads

SQuAD (Stanford Question Answering Dataset)

Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. Access it here.

PubMed Diabetes Dataset

The Pubmed Diabetes dataset consists of 19717 scientific publications from PubMed database pertaining to diabetes classified into one of three classes. The citation network consists of 44338 links. Each publication in the dataset is described by a TF/IDF weighted word vector from a dictionary which consists of 500 unique words. The README file in the dataset provides more details.

Download Link

Drug-Target Interaction Dataset

This dataset contains interactions between drugs and targets collected from DrugBank, KEGG Drug, DCDB, and Matador. It was originally collected by Perlman et al. It contains 315 drugs, 250 targets, 1,306 drug-target interactions, 5 types of drug-drug similarities, and 3 types of target-target similarities. Drug-drug similarities include Chemical-based, Ligand-based, Expression-based, Side-effect-based, and Annotation-based similarities. Target-target similarities include Sequence-based, Protein-protein interaction network-based, and Gene Ontology-based similarities. The original task on the dataset is to predict new interactions between drugs and targets based on different types of similarities in the network. Download link

Pharmacogenomics Datasets

PharmGKB data and knowledge is available as downloads. It is often critical to check with their curators at feedback@pharmgkb.org before embarking on a large project using these data, to be sure that the files and data they make available are being interpreted correctly. PharmGKB generally does NOT need to be a co-author on such analyses; They just want to make sure that there is a correct understanding of our data before lots of resources are spent.

Pancreatic Cancer Organoid Profiling

The dataset contains open RNA-Seq Gene Expression Quantification data and controlled WGS/WXS/RNA-Seq Aligned Reads, WXS Annotated Somatic Mutation, WXS Raw Somatic Mutation, and RNA-Seq Splice Junction Quantification. Documentation

AWS CLI Access (No AWS account required)

aws s3 ls s3://gdc-organoid-pancreatic-phs001611-2-open/ --no-sign-request

Africa Soil Information Service (AfSIS) Soil Chemistry

This dataset contains soil infrared spectral data and paired soil property reference measurements for georeferenced soil samples that were collected through the Africa Soil Information Service (AfSIS) project, which lasted from 2009 through 2018. Documentation

AWS CLI Access (No AWS account required)

aws s3 ls s3://afsis/ --no-sign-request

Dataset for Affective States in E-Environments

DAiSEE is the first multi-label video classification dataset comprising of 9068 video snippets captured from 112 users for recognizing the user affective states of boredom, confusion, engagement, and frustration “in the wild”. The dataset has four levels of labels namely – very low, low, high, and very high for each of the affective states, which are crowd annotated and correlated with a gold standard annotation created using a team of expert psychologists. Download it here.

NatureServe Explorer Dataset

NatureServe Explorer provides conservation status, taxonomy, distribution, and life history information for more than 95,000 plants and animals in the United States and Canada, and more than 10,000 vegetation communities and ecological systems in the Western Hemisphere.

The data available through NatureServe Explorer represents data managed in the NatureServe Central Databases. These databases are dynamic, being continually enhanced and refined through the input of hundreds of natural heritage program scientists and other collaborators. NatureServe Explorer is updated from these central databases to reflect information from new field surveys, the latest taxonomic treatments and other scientific publications, and new conservation status assessments. Explore Data here

Flight Records in the US

Airline On-Time Performance and Causes of Flight Delays – On_Time Data.

This database contains scheduled and actual departure and arrival times, reason of delay. reported by certified U.S. air carriers that account for at least one percent of domestic scheduled passenger revenues. The data is collected by the Office of Airline Information, Bureau of Transportation Statistics (BTS).

FlightAware.com has data but you need to pay for a full dataset.

The anyflights package supplies a set of functions to generate air travel data (and data packages!) similar to nycflights13. With a user-defined year and airport, the anyflights function will grab data on:

  • flights: all flights that departed a given airport in a given year and month
  • weather: hourly meterological data for a given airport in a given year and month
  • airports: airport names, FAA codes, and locations
  • airlines: translation between two letter carrier (airline) codes and names
  • planes: construction information about each plane found in flights

Airline On-Time Statistics and Delay Causes

The U.S. Department of Transportation’s (DOT) Bureau of Transportation Statistics (BTS) tracks the on-time performance of domestic flights operated by large air carriers. Summary information on the number of on-time, delayed, canceled and diverted flights appears in DOT’s monthly Air Travel Consumer Report, published about 30 days after the month’s end, as well as in summary tables posted on this website. BTS began collecting details on the causes of flight delays in June 2003. Summary statistics and raw data are made available to the public at the time the Air Travel Consumer Report is released. Access it here

Worldwide flight data

Open flights: As of January 2017, the OpenFlights Airports Database contains over 10,000 airports, train stations and ferry terminals spanning the globe

Download: airports.dat (Airports only, high quality)

Download: airports-extended.dat (Airports, train stations and ferry terminals, including user contributions)

Bureau of Transportation:

Flightera.net seems to have a lot of good data for free. It has in-depth data on flights and doesn’t seem limited by date. I can’t speak on the validity of the data though.

flightradar24.com has lots of data, also historically, they might be willing to help you get it in a nice format.

2019 Crime statistics in the USA

Dataset with arrest in US by race and separate states. Download Excel here

Stack Exchange
Infographic: Where People Are Most Willing to Donate Blood  | Statista You will find more infographics at Statista

Researchers from IBM, MIT and Harvard Announced The Release Of DARPA “Common Sense AI” Dataset Along With Two Machine Learning Models At ICML 2021

Building machines that can make decisions based on common sense is no easy feat. A machine must be able to do more than merely find patterns in data; it also needs a way of interpreting the intentions and beliefs behind people’s choices.

At the 2021 International Conference on Machine Learning (ICML), Researchers from IBM, MIT, and Harvard University have come together to release a DARPA “Common Sense AI” dataset for benchmarking AI intuition. They are also releasing two machine learning models that represent different approaches to the problem that relies on testing techniques psychologists use to study infants’ behavior to accelerate the development of AI exhibiting common sense. 

Source – Summary – Paper – IBM Blog

100 million protein structures Dataset by DeepMind

DeepMind creates ‘transformative’ map of human proteins drawn by AI. By the end of the year, DeepMind hopes to release predictions for 100 million protein structures, a dataset that will be “transformative for our understanding of how life works,

Here’s a good article about this topic

Google Dataset Search

Google Dataset Search

Malware traffic dataset

Comprises 1914081 records created from all malware traffic analysis .net PCAP files, from 2013 to 2021. The logs are generated using Suricata and Zeek.

Originator: ali_alwashali

Author: Here

 CPOST dataset on suicide attacks over four decades

The University of Chicago Project on Security and Threats presents the updated and expanded Database on Suicide Attacks (DSAT), which now links to Uppsala Conflict Data Program data on armed conflicts and includes a new dataset measuring the alliance and rivalry relationships among militant groups with connections to suicide attack groups. Access it here.

Credit Card Dataset – Survey of Consumer Finances (SCF) Combined Extract Data 1989-2019

 You can do a lot of aggregated analysis in a pretty straightforward way there.

Drone imagery with annotations for small object detection and tracking dataset

11 TB dataset of drone imagery with annotations for small object detection and tracking

Download and more information are available here

Dataset License: CDLA-Sharing-1.0

Helper scripts for accessing the dataset: DATASET.md

Dataset Exploration: Colab

NOAA High-Resolution Rapid Refresh (HRRR) Model

The HRRR is a NOAA real-time 3-km resolution, hourly updated, cloud-resolving, convection-allowing atmospheric model, initialized by 3km grids with 3km radar assimilation. Radar data is assimilated in the HRRR every 15 min over a 1-h period adding further detail to that provided by the hourly data assimilation from the 13km radar-enhanced Rapid Refresh.

Registry of Open Data on AWS

This registry exists to help people discover and share datasets that are available via AWS resources. Learn more about sharing data on AWS.

See all usage examples for datasets listed in this registry.

See datasets from Digital Earth AfricaFacebook Data for GoodNASA Space Act AgreementNIH STRIDESNOAA Big Data ProgramSpace Telescope Science Institute, and Amazon Sustainability Data Initiative.

Textbook Question Answering (TQA)

1,076 textbook lessons, 26,260 questions, 6229 images

Documentation: allenai.org/data/tqa

Download

Harmonized Cancer Datasets: Genomic Data Commons Data Portal

The GDC Data Portal is a robust data-driven platform that allows cancer
researchers and bioinformaticians to search and download cancer data for analysis.

Genomic Data Commons Data Portal
Genomic Data Commons Data Portal

The Cancer Genome Atlas

The Cancer Genome Atlas (TCGA), a collaboration between the National Cancer Institute (NCI) and National Human Genome Research Institute (NHGRI), aims to generate comprehensive, multi-dimensional maps of the key genomic changes in major types and subtypes of cancer.

AWS CLI Access (No AWS account required)

aws s3 ls s3://tcga-2-open/ --no-sign-request

Therapeutically Applicable Research to Generate Effective Treatments (TARGET)

The Therapeutically Applicable Research to Generate Effective Treatments (TARGET) program applies a comprehensive genomic approach to determine molecular changes that drive childhood cancers. The goal of the program is to use data to guide the development of effective, less toxic therapies. TARGET is organized into a collaborative network of disease-specific project teams.  TARGET projects provide comprehensive molecular characterization to determine the genetic changes that drive the initiation and progression of childhood cancers. The dataset contains open Clinical Supplement, Biospecimen Supplement, RNA-Seq Gene Expression Quantification, miRNA-Seq Isoform Expression Quantification, miRNA-Seq miRNA Expression Quantification data from Genomic Data Commons (GDC), and open data from GDC Legacy Archive. Access it here.

Genome Aggregation Database (gnomAD)

The Genome Aggregation Database (gnomAD) is a resource developed by an international coalition of investigators that aggregates and harmonizes both exome and genome data from a wide range of large-scale human sequencing projects. The summary data provided here are released for the benefit of the wider scientific community without restriction on use. Downloads

SQuAD (Stanford Question Answering Dataset)

Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. Access it here.

PubMed Diabetes Dataset

The Pubmed Diabetes dataset consists of 19717 scientific publications from PubMed database pertaining to diabetes classified into one of three classes. The citation network consists of 44338 links. Each publication in the dataset is described by a TF/IDF weighted word vector from a dictionary which consists of 500 unique words. The README file in the dataset provides more details.

Download Link

Drug-Target Interaction Dataset

This dataset contains interactions between drugs and targets collected from DrugBank, KEGG Drug, DCDB, and Matador. It was originally collected by Perlman et al. It contains 315 drugs, 250 targets, 1,306 drug-target interactions, 5 types of drug-drug similarities, and 3 types of target-target similarities. Drug-drug similarities include Chemical-based, Ligand-based, Expression-based, Side-effect-based, and Annotation-based similarities. Target-target similarities include Sequence-based, Protein-protein interaction network-based, and Gene Ontology-based similarities. The original task on the dataset is to predict new interactions between drugs and targets based on different types of similarities in the network. Download link

Pharmacogenomics Datasets

PharmGKB data and knowledge is available as downloads. It is often critical to check with their curators at feedback@pharmgkb.org before embarking on a large project using these data, to be sure that the files and data they make available are being interpreted correctly. PharmGKB generally does NOT need to be a co-author on such analyses; They just want to make sure that there is a correct understanding of our data before lots of resources are spent.

Pancreatic Cancer Organoid Profiling

The dataset contains open RNA-Seq Gene Expression Quantification data and controlled WGS/WXS/RNA-Seq Aligned Reads, WXS Annotated Somatic Mutation, WXS Raw Somatic Mutation, and RNA-Seq Splice Junction Quantification. Documentation

AWS CLI Access (No AWS account required)

aws s3 ls s3://gdc-organoid-pancreatic-phs001611-2-open/ --no-sign-request

Africa Soil Information Service (AfSIS) Soil Chemistry

This dataset contains soil infrared spectral data and paired soil property reference measurements for georeferenced soil samples that were collected through the Africa Soil Information Service (AfSIS) project, which lasted from 2009 through 2018. Documentation

AWS CLI Access (No AWS account required)

aws s3 ls s3://afsis/ --no-sign-request

Dataset for Affective States in E-Environments

DAiSEE is the first multi-label video classification dataset comprising of 9068 video snippets captured from 112 users for recognizing the user affective states of boredom, confusion, engagement, and frustration “in the wild”. The dataset has four levels of labels namely – very low, low, high, and very high for each of the affective states, which are crowd annotated and correlated with a gold standard annotation created using a team of expert psychologists. Download it here.

NatureServe Explorer Dataset

NatureServe Explorer provides conservation status, taxonomy, distribution, and life history information for more than 95,000 plants and animals in the United States and Canada, and more than 10,000 vegetation communities and ecological systems in the Western Hemisphere.

The data available through NatureServe Explorer represents data managed in the NatureServe Central Databases. These databases are dynamic, being continually enhanced and refined through the input of hundreds of natural heritage program scientists and other collaborators. NatureServe Explorer is updated from these central databases to reflect information from new field surveys, the latest taxonomic treatments and other scientific publications, and new conservation status assessments. Explore Data here

Flight Records in the US

Airline On-Time Performance and Causes of Flight Delays – On_Time Data.

This database contains scheduled and actual departure and arrival times, reason of delay. reported by certified U.S. air carriers that account for at least one percent of domestic scheduled passenger revenues. The data is collected by the Office of Airline Information, Bureau of Transportation Statistics (BTS).

FlightAware.com has data but you need to pay for a full dataset.

The anyflights package supplies a set of functions to generate air travel data (and data packages!) similar to nycflights13. With a user-defined year and airport, the anyflights function will grab data on:

  • flights: all flights that departed a given airport in a given year and month
  • weather: hourly meterological data for a given airport in a given year and month
  • airports: airport names, FAA codes, and locations
  • airlines: translation between two letter carrier (airline) codes and names
  • planes: construction information about each plane found in flights

Airline On-Time Statistics and Delay Causes

The U.S. Department of Transportation’s (DOT) Bureau of Transportation Statistics (BTS) tracks the on-time performance of domestic flights operated by large air carriers. Summary information on the number of on-time, delayed, canceled and diverted flights appears in DOT’s monthly Air Travel Consumer Report, published about 30 days after the month’s end, as well as in summary tables posted on this website. BTS began collecting details on the causes of flight delays in June 2003. Summary statistics and raw data are made available to the public at the time the Air Travel Consumer Report is released. Access it here

Worldwide flight data

Open flights: As of January 2017, the OpenFlights Airports Database contains over 10,000 airports, train stations and ferry terminals spanning the globe

Download: airports.dat (Airports only, high quality)

Download: airports-extended.dat (Airports, train stations and ferry terminals, including user contributions)

Bureau of Transportation:

Flightera.net seems to have a lot of good data for free. It has in-depth data on flights and doesn’t seem limited by date. I can’t speak on the validity of the data though.

flightradar24.com has lots of data, also historically, they might be willing to help you get it in a nice format.

2019 Crime statistics in the USA

Dataset with arrest in US by race and separate states. Download Excel here

The largest repository of standardized and structured statistical data

statisticaldatasets.data-planet.com/

Chess datasets

3.5 Million Chess Games

ML Dataset to practice methods of regression

Center for Machine Learning and Intelligent Systems

585 Data Sets

 

ManyTypes4Py: A benchmark Python Dataset for Machine Learning-Based Type Inference

  • The dataset is gathered on Sep. 17th 2020 from GitHub.
  • It has more than 5.2K Python repositories and 4.2M type annotations.
  • Use it to train  ML-based type inference model for Python
  • Access it here

Quadrature magnetoresistance in overdoped cuprates

Measurements of the normal (i.e. non-superconducting) state magnetoresistance (change in resistance with magnetic field) in several single crystalline samples of copper-oxide high-temperature superconductors. The measurements were performed predominantly at the High Field Magnet Laboratory (HFML) in Nijmegen, the Netherlands, and the Pulsed Magnetic Field Facility (LNCMI-T) in Toulouse, France. Complete Zip Download

The UMA-SAR Dataset: Multimodal data collection from a ground vehicle during outdoor disaster response training exercises

Collection of multimodal raw data captured from a manned all-terrain vehicle in the course of two realistic outdoor search and rescue (SAR) exercises for actual emergency responders conducted in Málaga (Spain) in 2018 and 2019: the UMA-SAR dataset. Full Dataset.

Child Mortality from Malaria

Child mortality numbers caused by malaria by country

Number of deaths of infants, neonatal, and children up to 4 years old caused by malaria by country from 2000 to 2015. Originator: World Health Organization

Child-Mortality-Numbers-by-Malaria-2015

Quora Question Pairs at Data.world

The dataset  will give anyone the opportunity to train and test models of semantic equivalence, based on actual Quora data. 400,000 lines of potential question duplicate pairs. Each line contains IDs for each question in the pair, the full text for each question, and a binary value that indicates whether the line truly contains a duplicate pair. Access it here.

MIMIC Critical Care Database

MIMIC is an openly available dataset developed by the MIT Lab for Computational Physiology, comprising deidentified health data associated with ~60,000 intensive care unit admissions. It includes demographics, vital signs, laboratory tests, medications, and more. Access it here.

Data.Gov: The home of the U.S. Government’s open data

Here you will find data, tools, and resources to conduct research, develop web and mobile applications, design data visualizations, and more. Search over 280000 Datasets.

Tidy Tuesday Dataset

TidyTuesday is built around open datasets that are found in the “wild” or submitted as Issues on our GitHub.

US Census Bureau: QuickFacts Dataset

QuickFacts provides statistics for all states and counties, and for cities and towns with a population of 5,000 or more.

Classical Abstract Art Dataset

Art that does not attempt to represent an accurate depiction of a visual reality but instead use shapes, colours, forms and gestural marks to achieve its effect

5000+ classical abstract art here, real artists with annotation. You can download them in very high resolution,  however you would have to crawl them first  with this scraper.

Interactive map of indigenous people around the world

Native-Land.ca is a website run by the nonprofit organization Native Land Digital. Access it here.

Data Visualization: A Wordcloud for each of the Six Largest Religions and their Religious Texts (Judaism, Christianity, and Islam; Hinduism, Buddhism, and Sikhism)

Highest altitude humans have been each year since 1961

Worldwide prevalence of drug use

Data Sciences - Top 400 Open Datasets - Data Visualization - Data Analytics - Big Data - Data Lakes

From the author:

I took the data from IHME’s Global Burden of Disease 2019 study (2019 all-ages prevalence of drug use disorders among both men and women for all countries and territories) and plotted it using R.

Also, what is going on in the US exactly? 3.3% of the population there is addicted and it’s the worst rate in the world.

World Population and Energy Consumption: History and Projections

Data Sciences - Top 400 Open Datasets - Data Visualization - Data Analytics - Big Data - Data Lakes

From the author:

I am working on a presentation and I found a similar graph but couldn’t source it, so I found the data and remade this:

Data:

Population 2010-2019: US Census Bureau via Alexa

Population 2020-2050: World Population Prospects – Population Division – United Nations

File POP/1-1: Total population (both sexes combined) by region, subregion and country, annually for 1950-2100 (thousands)Medium fertility variant, 2020 – 2100

World Energy Consumption: Annual Energy Outlook 2021

Number of Operational Nuclear Reactors by Region over Time

Data from Power Reactor Information System maintained by IAEA.

Countries with reactors in each region:

ASIA-E: China, Japan, South Korea, Taiwan

ASIA-S: India, Pakistan, Bangladesh (under construction)

EUROPE-E & ASIA-CTRL: Armenia, Bulgaria, Belarus, Czechia, Hungary, Kazakhstan, Lithuania, Romania, Russia, Slovenia, Slovakia, Ukraine, Turkey (under construction)

EUROPE-N, S & W: Belgium, Switzerland, Germany, Spain, Finland, France, UK, Italy, Netherlands, Sweden

LATIN AMERICA: Argentina, Brazil, Mexico

MIDDLE EAST: UAE, Iran

SUB-SAHARAN AFRICA: South Africa

USA & CANADA: Canada, USA (obviously)

Made with MATLAB

Edit: If it wasn’t clear, these are nuclear power stations only

source: r/dataisbeautiful

data.ohio.gov/wps/

National Household Travel Survey (US)

Conducted by the Federal Highway Administration (FHWA), the NHTS is the authoritative source on the travel behavior of the American public. It is the only source of national data that allows one to analyze trends in personal and household travel. It includes daily non-commercial travel by all modes, including characteristics of the people traveling, their household, and their vehicles. Access it here.

National Travel Survey (UK)

Statistics and data about the National Travel Survey, based on a household survey to monitor trends in personal travel.

The survey collects information on how, why, when and where people travel as well as factors affecting travel (e.g. car availability and driving license holding).

National Travel Survey data tables UK
National Travel Survey data tables UK

National Travel Survey (NTS)[Canada]

Monthly Railway Carloadings: Interactive Dashboard
Monthly Railway Carloadings: Interactive Dashboard

ENTUR: NeTEx or GTFS datasets [Norway]

NeTEx is the official format for public transport data in Norway and is the most complete in terms of available data. GTFS is a downstream format with only a limited subset of the total data, but we generate datasets for it anyway since GTFS can be easier to use and has a wider distribution among international public transport solutions. GTFS sets come in “extended” and “basic” versions. Access here.

The Swedish National Forest Inventory

A subset of the field data collected on temporary NFI plots can be downloaded in Excel format from this web site. The file includes a Read_me sheet and a sheet with field data from temporary plots on forest land1 collected from 2007 to 2019. Note that plots located on boundaries (for example boundaries between forest stands, or different land use classes) are not included in the dataset. The dataset is primarily intended to be used as reference data and validation data in remote sensing applications. It cannot be used to derive estimates of totals or mean values for a geographic area of any size. Download the dataset here

Large data sets from finance and economics applicable in related fields studying the human condition

World Bank Data: Countries Data | Topics Data | Indicators Data | Catalog

US Federal Statistics

Boards of Governors of the Federal Reserve: Data Download Program

CIA: The world Factbook provides basic intelligence on the history, people, government, economy, energy, geography, environment, communications, transportation, military, terrorism, and transnational issues for 266 world entities.

Human Development Report: United Nations Development Programme – Public Data Explorer

Consumer Price Index: The Consumer Price Index (CPI) is a measure of the average change over time in the prices paid by urban consumers for a market basket of consumer goods and services. Indexes are available for the U.S. and various geographic areas. Average price data for select utility, automotive fuel, and food items are also available.

Gapminder.org: Unveiling the beauty of statistics for a fact based world view Watch everyday life in hundreds of homes on all income levels across the world, to counteract the media’s skewed selection of images of other places.

Our world in Data: International Trade

Research and data to make progress against the world’s largest problems: 3139 charts across 297 topics, All free: open access and open source.

International Historical Statistics (by Brian Mitchell)

 
International Historical Statistics is a compendium of national and international socio-economic data from 1750 to 2010. Data are available in both Excel and PDF tabular formats. IHS is structured in three broad geographical divisions and ten themes: Africa / Asia / Oceania; The Americas and Europe. The database is structured in ten categories: Population and vital statistics; Labour force; Agriculture; Industry; External trade; Transport and communications; Finance; Commodity prices; Education and National accounts. Access here

World Input-Output Database

World Input-Output Tables and underlying data. World Input-Output Tables and underlying data, covering 43 countries, and a model for the rest of the world for the period 2000-2014. Data for 56 sectors are classified according to the International Standard Industrial Classification revision 4 (ISIC Rev. 4).

  • Data: Real and PPP-adjusted GDP in US millions of dollars, national accounts (household consumption, investment, government consumption, exports and imports), exchange rates and population figures.
  • Geographical coverage: Countries around the world
  • Time span: from 1950-2011 (version 8.1)
  • Available at: Online

Correlates of War Bilateral Trade

COW seeks to facilitate the collection, dissemination, and use of accurate and reliable quantitative data in international relations. Key principles of the project include a commitment to standard scientific principles of replication, data reliability, documentation, review, and the transparency of data collection procedures

  • Data: Total national trade and bilateral trade flows between states. Total imports and exports of each country in current US millions of dollars and bilateral flows in current US millions of dollars
  • Geographical coverage: Single countries around the world
  • Time span: from 1870-2009
  • Available at: Online here
  • This data set is hosted by Katherine Barbieri, University of South Carolina, and Omar Keshk, Ohio State University.

World Bank Open Data – World Development Indicators

Free and open access to global development data. Access it here.

World Trade Organization – WTO

The WTO provides quantitative information in relation to economic and trade policy issues. Its data-bases and publications provide access to data on trade flows, tariffs, non-tariff measures (NTMs) and trade in value added.

  • Data: Many series on tariffs and trade flows
  • Geographical coverage: Countries around the world
  • Time span: Since 1948 for some series
  • Available at: Online here
WTO - World Trade Organization
WTO – World Trade Organization

SMOKA Science Archive

The Subaru-Mitaka-Okayama-Kiso Archive, holds about 15 TB of astronomical data from facilities run by the National Astronomical Observatory of Japan. All data becomes publicly available after an embargo period of 12-24 months (to give the original observers time to publish their papers).

Graph Datasets

Multi-Domain Sentiment Dataset

The Multi-Domain Sentiment Dataset contains product reviews taken from Amazon.com from many product types (domains). Some domains (books and dvds) have hundreds of thousands of reviews. Others (musical instruments) have only a few hundred. Reviews contain star ratings (1 to 5 stars) that can be converted into binary labels if needed. Access it here.

A Global Database of Society

Supported by Google Jigsaw, the GDELT Project monitors the world’s broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, themes, sources, emotions, counts, quotes, images and events driving our global society every second of every day, creating a free open platform for computing on the entire world.

The Yahoo News Feed: Ratings and Classification Data

Dataset is 1.5 TB compressed, 13.5 TB uncompressed

Yahoo! Music User Ratings of Musical Artists, version 1.0 (423 MB)

This dataset represents a snapshot of the Yahoo! Music community’s preferences for various musical artists. The dataset contains over ten million ratings of musical artists given by Yahoo! Music users over the course of a one month period sometime prior to March 2004. Users are represented as meaningless anonymous numbers so that no identifying information is revealed. The dataset may be used by researchers to validate recommender systems or collaborative filtering algorithms. The dataset may serve as a testbed for matrix and graph algorithms including PCA and clustering algorithms. The size of this dataset is 423 MB.
 

Yahoo! Movies User Ratings and Descriptive Content Information, v.1.0 (23 MB)

This dataset contains a small sample of the Yahoo! Movies community’s preferences for various movies, rated on a scale from A+ to F. Users are represented as meaningless anonymous numbers so that no identifying information is revealed. The dataset also contains a large amount of descriptive information about many movies released prior to November 2003, including cast, crew, synopsis, genre, average ratings, awards, etc. The dataset may be used by researchers to validate recommender systems or collaborative filtering algorithms, including hybrid content and collaborative filtering algorithms. The dataset may serve as a testbed for relational learning and data mining algorithms as well as matrix and graph algorithms including PCA and clustering algorithms. The size of this dataset is 23 MB.
 

Yahoo News Video dataset, version 1.0 (645MB)

The dataset is a collection of 964 hours (22K videos) of news broadcast videos that appeared on Yahoo news website’s properties, e.g., World News, US News, Sports, Finance, and a mobile application during August 2017. The videos were either part of an article or displayed standalone in a news property. Many of the videos served in this platform lack important metadata, such as an exhaustive list of topics associated with the video. We label each of the videos in the dataset using a collection of 336 tags based on a news taxonomy designed by in-house editors. In the taxonomy, the closer the tag is to the root, the more generic (topically) it is.
etc…

Other Datasets

More than 1 TB

  • The 1000 Genomes project makes 260 TB of human genome data available
  • The Internet Archive is making an 80 TB web crawl available for research 
  • The TREC conference made the ClueWeb09 [3] dataset available a few years back. You’ll have to sign an agreement and pay a nontrivial fee (up to $610) to cover the sneakernet data transfer. The data is about 5 TB compressed.
  • ClueWeb12  is now available, as are the Freebase annotations, FACC1 
  • CNetS at Indiana University makes a 2.5 TB click dataset available 
  • ICWSM made a large corpus of blog posts available for their 2011 conference. You’ll have to register (an actual form, not an online form), but it’s free. It’s about 2.1 TB compressed. The dataset consists of over 386 million blog posts, news articles, classifieds, forum posts and social media content between January 13th and February 14th. It spans events such as the Tunisian revolution and the Egyptian protests (see http://en.wikipedia.org/wiki/January_2011 for a more detailed list of events spanning the dataset’s time period). Access it here
  • The Yahoo News Feed dataset is 1.5 TB compressed, 13.5 TB uncompressed
  • The Proteome Commons makes several large datasets available. The largest, the Personal Genome Project , is 1.1 TB in size. There are several others over 100 GB in size.

More than 1 GB

Power and Energy Consumption Open Datasets

These data are intended to be used by researchers and other professionals working in power and energy related areas and requiring data for design, development, test, and validation purposes. These data should not be used for commercial purposes.

The Million Playlist Dataset (Spotify)

A dataset and open-ended challenge for music recommendation research ( RecSys Challenge 2018). Sampled from the over 4 billion public playlists on Spotify, this dataset of 1 million playlists consist of over 2 million unique tracks by nearly 300,000 artists, and represents the largest public dataset of music playlists in the world. Access it here

How much each of 20 most popular artists earns from Spotify.

How much each of 20 most popular artists earns from Spotify.
How much each of 20 most popular artists earns from Spotify.

Regression Analysis Cheat Sheet

Hotel Reviews Dataset from Yelp

20k+ Hotel Reviews from Yelp for 5 Star Hotels in Las Vegas.

This dataset can be used for the following applications and more:

Analyzing trends,  Sentiment Analysis / Opinion Mining, Sentiment Analysis / Opinion Mining, Competitor Analysis. Access it here.

A truncated version with 500 reviews is also available on Kaggle here

Motorcycle Crash data

1- Texas: Perform specific queries and analysis using Texas traffic crash data.

2- BTS: Motorcycle Rider Safety Data

3- National Transportation Safety Board: US Transportation Fatalities in 2019

4- Fatal single vehicle motorcycle crashes

5- Motorcycle crash causes and outcomes : pilot study

6- Motorcycle Crash Causation Study: Final Report

Download a collection of news articles relating to natural disasters over an eight-month period. Access it here.

World Population Data by Country and Age Group

1- WorldoMeter: Countries in the world by population (2021)

2- Worldometer: Current World Population Live

Top 10 richest billionaires from 1987-2021

Top 10 Richest People in the world

Source: Here

World’s Top 80 Wealthiest People by Country of Origin as of November 2021

Source: Here

From the author:

Needless to say, the United States absolutely dominates this list more than any other country. 9 of the top 10 are Americans, you’d have to combine the next 5 countries after the US to match their output of 33 among the top 80, and you’d have to combined every other country not named China on this graph to equal the USA.

To break things down based on region:

– The Americas has 34 individuals on this list with USA (33) and Mexico (1)

– Asia-Pacific has 28 individuals on this list with China (14), India (5), Hong Kong (4), Japan (3), and Australia (2)

– Europe has 18 individuals on this list with France (5), Russia (5), Germany (3), Italy (2), UK (1), Ireland (1), and Spain (1)

How Americans Spend Money on Halloween

How Americans Spend Money on Halloween

Source: here

How the Duration of an Average World Series Baseball Game Has Changed Over 118 Years

r/dataisbeautiful - [OC] How the Duration of an Average World Series Baseball Game Has Changed Over 118 Years

Source: Here

Investment-Related Dataset with both Qualitative and Quantitative Variables

1- Numer.ai:  Anonymized and feature normalized financial data which is interesting for machine learning applications. Download here

2- Snowflake Data Marketplace: Snowflake Data Marketplace gives data scientists, business intelligence and analytics professionals, and everyone who desires data-driven decision-making, access to more than 375 live and ready-to-query data sets from more than 125 third-party data providers and data service providers

3- Quandl: The premier source for financial, economic and alternative datasets, serving investment professionals.

National Obesity Monitor

The National Health and Nutrition Examination Survey (NHANES) is conducted every two years by the National Center for Health Statistics and funded by the Centers for Disease Control and Prevention. The survey measures obesity rates among people ages 2 and older. Find the latest national data and trends over time, including by age group, sex, and race. Data are available through 2017-2018, with the exception of obesity rates for children by race, which are available through 2015-2016. Access here

State of Childhood Obesity
State of Childhood Obesity

The World’s Nations by Fertility Rate 2021

The world nation 's fertility rates
The world’s nations fertility rates

Total number of deaths due to Covid19 vis-à-vis Population in million

Total number of deaths due to Covid19 vis-à-vis Population in million
Total number of deaths due to Covid19 vis-à-vis Population in million

 Unlike its successor (COVID), SARS only heavily impacted 5 countries.

r/dataisbeautiful - [OC] Unlike its successor (COVID), SARS only heavily impacted 5 countries.

USA Cigarettes Sold v. Lung Cancer Death Rates

r/dataisbeautiful - USA Cigarettes Sold v. Lung Cancer Death Rates [OC]

Google searches for different emotions during each hour of the day and night

Google searches for different emotions during each hour of the day and night
Google searches for different emotions during each hour of the day and night

Where do the world’s CO2 emissions come from? This map shows emissions during 2019. Darker areas indicate areas with higher emissions

Where do the world's CO2 emissions come from? This map shows emissions during 2019. Darker areas indicate areas with higher emissions
Where do the world’s CO2 emissions come from? This map shows emissions during 2019. Darker areas indicate areas with higher emissions

Global Linguistic Diversity

Global Linguistic Diversity
Global Linguistic Diversity

Where in the world are the densest forests? Darker areas represent higher density of trees.

Where in the world are the densest forests? Darker areas represent higher density of trees.
Where in the world are the densest forests? Darker areas represent higher density of trees.

Likes and Dislikes per movie genre

Like and Dislike per movie genre
Like and Dislike per movie genre

Global Historical Climatology Network-Monthly (GHCN-M) temperature dataset

NCEI first developed the Global Historical Climatology Network-Monthly (GHCN-M) temperature dataset in the early 1990s. Subsequent iterations include version 2 in 1997, version 3 in May 2011, and version 4 in October 2018.

Are there any places where the climate is recently getting colder?
Are there any places where the climate is recently getting colder?

Electric power consumption (kWh per capita)

The World’s Most Eco-Friendly Countries

Alternate Source from Wikipedia : List of countries by carbon dioxide emissions per capita

List of countries by carbon dioxide emissions per capita
List of countries by carbon dioxide emissions per capita
Worldwide CO2 Emission
Worldwide CO2 Emission

Alcohol-Impaired Driving Deaths by State & County [US]

Alcohol Impaired Driving by State
Alcohol Impaired Driving by State

Alcohol Impaired driving by counties
Alcohol Impaired driving by county

% change in life expectancy from 2020 to 2021 across the globe

% change in life expectancy from 2020 to 2021 across the globe
% change in life expectancy from 2020 to 2021 across the globe

This is how life expectancy is calculated.

How Many Years Till the World’s Reserves Run Out of Oil?

How Many Years Till the World's Reserves Run Out of Oil?
How Many Years Till the World’s Reserves Run Out of Oil?

Data Source Here: Note that these values can change with time based on the discovery of new reserves, and changes in annual production.

Which energy source has the least disadvantages?

How many People Did Nuclear Energy Kill?

Here’s a paper on the wind fatalities

ipcc.ch/site/assets/

Human development index (HDI) by world subdivisions

Human development index (HDI) by world subdivisions
Human development index (HDI) by world subdivisions

The Human Development Index (HDI) is a statistic composite index of life expectancy, education (mean years of schooling completed and expected years of schooling upon entering the education system), and per capita income indicators, which are used to rank countries into four tiers of human development.

Data sourcesubnational human development index website 

US Streaming Services Market Share, 2020 vs 2021

US Streaming Services Market Share, 2020 vs 2021
US Streaming Services Market Share, 2020 vs 2021

Number of tweets deleted by month

Number of tweets deleted by month in 2020
Number of tweets deleted by month in 2020

Tweet Deleter

Average Career Length by Sports Profession

Source: r/dataisbeautiful

From the author:

Got these numbers from here

Numbers like these are a quick reminder that not every athlete is LeBron James or Roger Federer who can play their sport at such high levels for their entire young adulthood while becoming billionaires in the process. Many careers are short lived and end abruptly while the athlete is still very young and some don’t really have a plan B.

NFL being at the bottom here doesn’t surprise me though as most positions (with the exception of QB and kicker) in US Football is lowkey bodily suicide.

Football/Soccer Leagues with the fairest distributions of money have seen the most growth in long-term global interest.

Football Leagues with the fairest distributions of money have seen the most growth in long-term global interest.
Football Leagues with the fairest distributions of money have seen the most growth in long-term global interest.

How Much Does Your Favorite Fast Food Brand Spend on Ads?

Sources:

mcdonald-s-advertising-spending-worldwide/

ad-spend-subway-usa/

dominos-pizza-advertising-spending-usa/

ad-spend-wednys-usa/

ad-spend-burger-king-usa/

advertising-expense-chick-fil-a/

starbucks-advertising-spending-in-the-us

Historical population count of Western Europe

Results from survey on how to best reduce your personal carbon footprint

Results from survey on how to best reduce your personal carbon footprint
Results from survey on how to best reduce your personal carbon footprint

Data from IpsosMori

Where does the world’s non-renewable energy come from? 

r/dataisbeautiful - Where does the world's non-renewable energy come from? Zoom in to see a point for each power plant! [OC]

The data comes from the Global Power Plant Database. The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights for one’s own analysis. The database covers approximately 30,000 power plants from 164 countries and includes thermal plants (e.g. coal, gas, oil, nuclear, biomass, waste, geothermal) and renewables (e.g. hydro, wind, solar). Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. It will be continuously updated as data becomes available.

Recorded Music Industry Revenues from 1997 to 2020

Source: riaa.com/

US Trade Surpluses and Deficits by Country (2020)

https://www.reddit.com/r/dataisbeautiful/comments/n446a3/oc_us_trade_surpluses_and_deficits_by_country_2020/?utm_source=share&utm_medium=web2x&context=3

Facebook Monthly Active Users

Facebook data is based on the end of year from 2004 to 2020

Facebook monthly active users

Source: SeeMetrics.com

Heat map of the past 50,000 earthquakes pulled from USGS sorted by magnitude

Source:  USGS website

Where do the world’s methane (CH4)emissions come from?

Darker areas indicate areas with higher emissions.

Source: Data comes from EDGARv5.0 website and Crippa et al. (2019)

Earth Surface Albedo (1950 to 2020)

Data Source: ECMWF ERA5

Wealth of Forbes’ Top 100 Billionaires vs All Households in Africa

https://www.reddit.com/r/dataisbeautiful/comments/n1su36/oc_animated_wealth_of_forbes_top_100_billionaires/?utm_source=share&utm_medium=web2x&context=3
Sources:
Forbes’ 35th Annual World’s Billionaires List
Credit Suisse Global Wealth Report 2020
United Nations World Population Prospects

Forbes Billionaires list

United nations world population prospects

Credit Suisse Global Wealth Report 2020

20 years of Apple sales in a minute

https://www.reddit.com/r/dataisbeautiful/comments/n17ctc/oc_20_years_of_apple_sales_in_a_minute/?utm_source=share&utm_medium=web2x&context=3
Source: Apple’s quarterly and annual financial filings with the SEC over the last 20 years

Source: Wikipedia

Racial Diversity of Each State (Based on US Census 2019 Estimates)

r/dataisbeautiful - [OC] Racial Diversity of Each State (Based on US Census 2019 Estimates)

Computation:

Suppose your state is 60% orc, 30% undead, and 10% tauren. You chance in a random selection of two being of the same race is as follows:

  • 36% chance ((60%)2) of two orcs

  • 9% chance ((30%)2) of two undead

  • 1% chance ((10%)2) of two tauren

For a total of 46%. The diversity index would be 100% minus that, or 54%.

Race and Ethnicity in the US

A curated, daily feed of newly published datasets in machine learning

Machine Learning: CIFAR-10 Dataset

A curated, daily feed of newly published datasets in machine learning

The CIFAR-10 dataset consists of 60000 32×32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

Machine Learning: ImageNet

The ImageNet dataset contains 14,197,122 annotated images according to the WordNet hierarchy. Since 2010 the dataset is used in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), a benchmark in image classification and object detection. The publicly released dataset contains a set of manually annotated training images.

Machine Learning: The MNIST Database of Handwritten Digits

The MNIST database of handwritten digits, available from this page, has a training set of 60,000 examples, and a test set of 10,000 examples. It is a subset of a larger set available from NIST. The digits have been size-normalized and centered in a fixed-size image.

It is a good database for people who want to try learning techniques and pattern recognition methods on real-world data while spending minimal efforts on preprocessing and formatting. Access it here.

The Massively Multilingual Image Dataset (MMID)

MMID is a large-scale, massively multilingual dataset of images paired with the words they represent collected at the University of Pennsylvania. The dataset is doubly parallel: for each language, words are stored parallel to images that represent the word, and parallel to the word’s translation into English (and corresponding images.) . Dcumentation.

AWS CLI Access (No AWS account required)

aws s3 ls s3://mmid-pds/ --no-sign-request

AWS Azure Google Cloud Cloud Certification Exam Prep App
AWS Azure Google Cloud Cloud Certification Exam Prep App: AWS CCP Cloud Practitioner CLF-C01, AWS Solution Architect Associate SAA-C02, AWS Developer Associate DEV-C01, AWS DAS-C01, Azure Fundamentals AZ900, Azure Administrator AZ104, Google Associate Cloud Engineer, AWS Specialty Data Analytics DAS-C01, AWS and Google Professional Machine Learning Specialty MLS-C01

Capitol insurrection arrests per million people by state

How have cryptocurrencies done during the Pandemic?

Data Source: Downloaded performance data on these cryptocurrencies from Investing.com which provides free historic data

Share of US Wealth by Generation

r/dataisbeautiful - Share of US Wealth by Generation [OC]

Source: US Federal Reserve

Top 100 Cryptocurrencies by Market Cap

Top 100 Cryptocurrencies by Market Cap

Data Source from coinmarketcap.com/

 Crypto race: DOGE vs BTC, last 365 days

Data sources: Coindesk BTC, Coindesk Dodge

 Yearly Performance of TOP 100 cryptocurrencies
Yearly Performance of TOP 100 cryptocurrencies

12,000 years of human population dynamics

Countries with a higher Human Development Index (HDI) than the European Union (EU)

HDI is calculated by the UN every year to measure a country’s development using average life expectancy, education level, and gross national income per capita (PPP). The EU has a collective HDI of 0.911.

Data Source: Here

Countries with a higher Human Development Index (HDI) than the United States (US)

Data source: Human Development Report 2020

Child marriage by country, by gender

Data on the percentage of children married before reaching adulthood (18 years).

Data source The State of the World’s Children 2019

 

Wars with greater than 25,000 deaths by year

Data Source : Wikipedia

Population Projection for China and India till 2050

Data Source: Here

Relative cumulative and per capita CO2 emissions 1751-2017

 

Relative cumulative and per capita CO2 emissions 1751-2017
Relative cumulative and per capita CO2 emissions 1751-2017

Dat Source: ourworldindata.org

Formula 1 Cumulative Wins by Team (1950-2021)

Data Source : f1-fansite.com/f1-results/

Countries with the most nuclear warheads. A couple of days ago I posted this with a logarithmic scale.

Data source: Wikipedia

Using machine learning methods to group NFL quarterbacks into archetypes

Using machine learning methods to group NFL quarterbacks into archetypes
Using machine learning methods to group NFL quarterbacks into archetypes

Data Source:

Data collected from a  series of rushing and passing statistics for NFL Quarterbacks from 2015-2020 and performed a machine learning algorithm called clustering, which automatically sorts observations into groups based on shared common characteristics using a mathematical “distance metric.”

The idea was to use machine learning to determine NFL Quarterback Archetype to agnostically determine which quarterbacks were truly “mobile” quarterbacks, and which were “pocket passers” that relied more on passing. I used a number of metrics in my actual clustering analysis, but they can be effectively summarized across two dimensions: passing and rushing, which can be further roughly summarized across two metrics: passer rating and rushing yards per year. Plotting the quarterbacks along these dimensions and plotting the groups chosen by the clustering methodology shows how cleanly the methodology selected the groups.

Read this blog article on the process for more information if you’re interested, or just check out this blog in general if you found this interesting!

Data: Collected from the ESPN API

2M rows of 1-min S&P bars (12 years of stock data) – 2008-2021

Intraday Stock Data (1 min) – S&P 500 – 2008-21: 12 years of 1 minute bars for data science / machine learning.

Granular stock bar data for research is difficult to find and expensive to buy. The author has compiled this library from a variety of sources and is making it available for free.

One compressed CSV file with 9 columns and 2.07 million rows worth of 1 minute SPY bars.  Access it here

A global database of COVID-19 vaccinations

Cumulative number of COVID-19 doses administered by country.
Cumulative number of COVID-19 doses administered by country.
COVID-19 vaccine doses administered per 100 people versus gross domestic product per capita.
COVID-19 vaccine doses administered per 100 people versus gross domestic product per capita.
Timeline of innovation in the development of vaccines.
Timeline of innovation in the development of vaccines.

Datasets: A live version of the vaccination dataset and documentation are available in a public GitHub repository here. These data can be downloaded in CSV and JSON formats. PDF.

 A list of available datasets for machine learning in manufacturing

Industrial ML Datasets: curated list of datasets, publicly available for machine learning researches in the area of manufacturing.

Predictive Maintenance and Condition Monitoring

NameYearFeature TypeFeature CountTarget VariableInstancesOfficial Train/Test SplitData SourceFormat
Diesel Engine Faults Features2020Signal84C (4)3.500 SyntheticMATLink

Process Monitoring

NameYearFeature TypeFeature CountTarget VariableInstancesOfficial Train/Test SplitData SourceFormat 
High Storage System Anomaly Detection2018Signal20C (2)91.000 SyntheticCSVLink

Predictive Quality and Quality Inspection

NameYearFeature TypeFeature CountTarget VariableInstancesOfficial Train/Test SplitData SourceFormat 
Casting Product Quality Inspection2020Image300×300
512×512
C (2)7.348✔️RealJPGLink

Process Parameter Optimization

NameYearFeature TypeFeature CountInstancesOfficial Train/Test SplitData SourceFormat 
Laser Welding2020Signal13361 RealXLSLink

Data Analytics Certification Questions and Answers Dumps

Datasets needed for Crop Disease Identification using image processing

Here is a collection of datasets with images of leaves

and more generic image datasets that include plant leaves

http://visualgenome.org/

http://image-net.org/

Plant Phenotyping

One hundreds plant species datasets

cvonline 

A Database of Leaf Images: Practice towards Plant Conservation with Plant Pathology

Survival Analysis datasets for machines

Download it here

English alphabet organized by each letter’s note in ABC

English alphabet organised by each letter's note in ABC
English alphabet organized by each letter’s note in ABC

Discover datasets hosted in thousands of repositories across the Web using datasetsearch.research.google.com

#dataset #search   @Google

Create, maintain, and contribute to a long-living dataset that will update itself automatically across projects.

Datasets should behave like git repositories.

Image

Datasets should behave like git repositories
Datasets should behave like git repositories

Learn how to create, maintain, and contribute to a long-living dataset that will update itself automatically across projects, using git and DVC as versioning systems, and DAGsHub as a host for the datasets. 

Human Rights Measurement Initiative Datasets

Image

World Wide Energy Production by Source 1860 – 2019

[OC] World Wide Energy Production by Source 1860 – 2019 from dataisbeautiful

Data source: ourworldindata.org/energy

 Project Sunroof – Solar Electricity Generation Potential by Census Tract/Postal Code

 Courtesy of Google’s Project Sunroof: This dataset essentially describes the rooftop solar potential for different regions, based on Google’s analysis of Google Maps data to find rooftops where solar would work, and aggregate those into region-wide statistics.

It comes in a couple of aggregation flavors – by census tract , where the region name is the census tract id, and by postal code , where the name is the postal code. Each also contains latitude/longitude bounding boxes and averages, so that you can download based on that, and you should be able to do custom larger aggregations using those, if you’d like.

Carbon emission arithmetic + hard v. soft science

carbon emission arithmetic + hard v. soft science [oc] from dataisbeautiful

Data sources: Video From data-driven documentary The Fallen of World War II. Here and Here

Most popular Youtuber in every country 2021

What Does 1GB of Mobile Data Cost in Every Country?

What Does 1GB of Mobile Data Cost in Every Country?

Key Concepts of Data Science

A large dataset aimed at teaching AI to code, it consists of some 14M code samples and about 500M lines of code in more than 55 different programming languages, from modern ones like C++, Java, Python, and Go to legacy languages like COBOL, Pascal, and FORTRAN.

GitHub repo:

Download page

NSRDB: National Solar Radiation Database

 Download instructions are here

Cheat Sheet for Machine Learning, Data Science.

No photo description available.
Cheat Sheet for Machine Learning and Data Science

Emigrants from the UK by Destination

 

r/dataisbeautiful - [OC] Emigrants from the UK by Destination

Data source: Originally at the location marked on the Sankey Flow but is now here

 

Direct link to the spreadsheet used

US Rivers and Streams Dataset

Data source: hub.arcgis.com/

Data visualization

r/dataisbeautiful - [OC] US Rivers and Streams

Bubble Chart that compares the GDP of the G20 Countries

https://www.reddit.com/r/dataisbeautiful/comments/ndsjrz/oc_bubble_chart_that_compares_the_gdp_of_the_g20/?utm_source=share&utm_medium=web2x&context=3

Data source: databank.worldbank.org

Desktop OS Market Share 2003 – 2021

[OC] Desktop OS Market Share 2003 – 2021 from dataisbeautiful

Data source: w3school

National Parks of North America

r/dataisbeautiful - [OC] National Parks of North America

Data Source: DataBayou

 NPS.gov, Open.canada.ca, and sig.conanp.gob.mx 

 

Inflation of Bitcoin and DogeCoin vs. Federal Reserve target

r/dataisbeautiful - [OC] Inflation of Bitcoin and DogeCoin vs. Federal Reserve target

Data source:

Percentage of women who experienced physical or sexual violence since the age of 15 in the EU

r/dataisbeautiful - Percentage of women who experienced physical or sexual violence since the age of 15 in the EU

Data Source from The Guardian: 

The whole report –  Questionnaire

Canadian Interprovincial Migration

Canadian Interprovincial Migration
Canadian Interprovincial Migration

Some context  here

Data  scraped from StatsCan

 

Covid-19 Vaccination Doses Administered per 100 in the G20

Data source: ourworldindata.org covid-vaccinations

What does per 100 mean?

When the whole country is double vaccinated, the value will be 200 doses per 100 population. At the moment the UK is like 85, which is because ~70% of the population has had at least one dose and ~15% of the population (which is a subset of that 70%) have had two. Hence ~30% are currently unprotected – myself included until Sunday.

Import/Export of Conventional Arms by Different Countries over past 2 decades

DataSource: SIPRI Arms Transfer Database

Aggregated disease comparison dataset – Ensemble de données agrégées de comparaison des maladies

Data Source: Here and Here

According to the author of the source data: “For the 1918 Spanish Flu, the data was collected by knowing that the total counts were 500M cases and 50M deaths, and then taking a fraction of that per day based on the area of this graph image:” – the graph is used is here:

Visualización y conjunto de datos de comparación de enfermedades agregadas

Trending Google Searches by State Between 2018 and 2020 – Tendances des recherches Google par État

Data source: trends.google.com Trending topics from 2010 to 2019 were taken from Google’s annual Year in Search summary 2010-2029

The full, ~11 minute video covering the whole 2010s decade is available here at youtu.be/xm91jBeN4oo

Google Trends provides weekly relative search interest for every search term, along with the interest by state. Using these two datasets for each term, we’re able to calculate the relative search interest for every state for a particular week. Linear interpolation was used to calculate the daily search interest.

 

Market capitalization in billion dollars of Top 20 Cryptocurrencies in 2021-05-20 – crypto-monnaies

Data source: CoinMarket from end of 2013 until present

Capitalisation boursière en milliards de dollars des 20 principales crypto-monnaies en 2021-05-20

 

Top Chess Players From 2000-2020, Meilleurs joueurs d’échecs,  Лучшие шахматисты с 2000 по 2020 год

Data source: ratings.fide.com/

The y-axis is the world elo ratings (called FIDE ratings).

Comparing Emissions Sources – How to Shrink your Carbon Footprint More Effectively

r/dataisbeautiful - [OC] Comparing Emissions Sources - How to Shrink your Carbon Footprint More Effectively

 Data sources: Here

Source article: Here

Oil and gas-fired power plants in the world –

La dependencia de los combustibles fósiles – La dépendance aux énergies fossiles – 

r/dataisbeautiful - [OC] Oil and gas-fired power plants in the world

Data is from the Global Power Plant Database (World Resources Institute)

See map’s description here

Plantas de electricidad que funcionan con gas y petróleo

Top 100 Reddit posts of all time

r/dataisbeautiful - [OC] I recently made a graph showing the Top 100 Reddit posts of all time. Some people said I should make a pie chart too, so here it is!

Source: r/all on Reddit

Tool used: meta-chart.com

Fastest routes on land (and sometimes, boat) between all 990 pairs of European capitals

Las rutas más rápidas en tierra (y, a veces, en barco) entre los 990 pares de capitales europeas

Les itinéraires les plus rapides sur terre (et parfois en bateau) entre les 990 paires de capitales européennes

Source: Reddit

From the author: I started with data on roads from naturalearth.com, which also includes some ferry lines. I then calculated the fastest routes (assuming a speed of 90 km/h on roads, and 35 km/h on boat) between each pair of 45 European capitals. The animation visualizes these routes, with brighter lines for roads that are more frequently “traveled”.

In reality these are of course not the most traveled roads, since people don’t go from all capitals to all other capitals in equal measure. But I thought it would be fun to visualize all the possible connections.

The model is also very simple, and does not take into account varying speed limits, road conditions, congestion, border checks and so on. It is just for fun!

In order to keep the file size manageable, the animation only shows every tenth frame.

Is Russia, Turkey or country X really part of Europe? That of course depends on the definition, but it was more fun to include them than to exclude them! The Vatican is however not included since it would just be the same as the Rome routes. And, unfortunately, Nicosia on Cyprus is not included to due an error on my behalf. It should be!

Link to final still image in high resolution on my twitter

 

Pokemon Dataset

  1. Dataset of all 825 Pokemon (this includes Alolan Forms). It would be preferable if there are at least 100 images of each individual Pokemon.

pokedex: This is a Python library slash pile of data containing a whole lot of data scraped from Pokémon games. It’s the primary guts of veekun.

pokeapi.co/about

2) This dataset comprises of more than 800 pokemons belonging up to 8 generations.

Using this dataset have been fun for me. I used it to create a mosaic of pokemons taking image as reference. You can find it here and it’s free to use: Couple Mosaic (powered by Pokemons)

Here is the data type information in the file:

  • Name: Pokemon Name
  • Type: Type of Pokemon like Grass / Fire / Water etc..,.
  • HP: Hit Points
  • Attack: Attack Points
  • Defense: Defence Points
  • Sp. Atk: Special Attack Points
  • Sp. Def: Special Defence Points
  • Speed: Speed Points
  • Total: Total Points
  • url: Pokemon web-page
  • icon: Pokemon Image

Data File: Pokemon-Data.csv

30×30 m Worldwide High-Resolution Population and Demographics Data

ETL pipeline for Facebook’s research project to provide detailed large-scale demographics data. It’s broken down in roughly 30×30 m grid cells and provides info on groups by age and gender.

Population Density Overview

Data Source and API for access

Article about Dataset at Medium

Gridded global datasets for Gross Domestic Product and Human Development Index over 1990–2015

Rasterized GDP dataset – basically a heat map of global economic activity.

Gap-filled multiannual datasets in gridded form for Gross Domestic Product (GDP) and Human Development Index (HDI)

Data source here:

Decrease in worldwide infant mortality from 1950 to 2020

Post image

Data Sources: United Nations, CIA World Factbook, IndexMundi.

Data Collectors

Data Unblockers

Countries of the world sorted by those that have warmed the most in the last 10 years, showing temperatures from 1890 to 2020 

r/dataisbeautiful - Countries of the world sorted by those that have warmed the most in the last 10 years, showing temperatures from 1890 to 2020 [OC]

Data source: Gistemp temperature data

The GISS Surface Temperature Analysis ver. 4 (GISTEMP v4) is an estimate of global surface temperature change. Graphs and tables are updated around the middle of every month using current data files from NOAA GHCN v4 (meteorological stations) and ERSST v5 (ocean areas), combined as described in our publications Hansen et al. (2010) and Lenssen et al. (2019).

Climate change concern vs personal spend to reduce climate change

r/dataisbeautiful - [OC] Climate change concern vs personal spend to reduce climate change

Data Source: Competitive Enterprise Institute (PDF)

 Less than 20 firms produce over a third of all carbon emissions

The Illusion of Choice in Consumer Brands

The Illusion of Choice in Consumer Brands

Buying a chocolate bar? There are seemingly hundreds to choose from, but its just the illusion of choice. They pretty much all come from Mars, Nestlé, or Mondelēz (which owns Cadbury).

Source: Visual Capitalist

Yearly Software Sales on PlayStation Consoles since 1994

r/dataisbeautiful - [OC] Yearly Software Sales on PlayStation Consoles since 1994

Some context for these numbers :

  • PS4 holds the record for being the console to have sold the most games in video game history (> 1.622B units)
    • Previous record holder was PS2 at 1.537B games sold
  • PS4 holds the record for having sold the most games in a single year (> 300M units in FY20)
  • FY20 marks the biggest yearly software sales in PlayStation ecosystem with more than 338M units
  • Since PS5 release, Sony starts combining PS4/PS5 software sales
  • In FY12, Sony combined PS2/PS3 and PSP/VITA software sales
  • Sony stopped disclosing software sales in FY13/14

Yearly Hardware Sales of PlayStation Consoles since 1994

r/dataisbeautiful - [OC] Yearly Hardware Sales of PlayStation Consoles since 1994

Sony combined PS2/PS3 hardware sales in FY12 and combined PSP/VITA sales in FY12/13/14

Cybertruck vs F150 Lightning pre-orders, by time since debut

r/dataisbeautiful - [OC] Cybertruck vs F150 Lightning pre-orders, by time since debut

Source: Ford exec tweeting about preorder numbers this week

Top 100 Most Populous City Proper in the world

r/dataisbeautiful - (Fixed once again) Top 100 Most Populous City Proper in the world. [OC]

The City with 32 million is Chongqing, Shan is Shanghai, Beijin is Beijing, and Guangzho is Guangzhou

 

Tax data for different countries

Dataset is here

What do Europeans feel most attached to – their region, their country, or Europe?

r/dataisbeautiful - [OC] What do Europeans feel most attached to - their region, their country, or Europe?

Data source: Builds on data from the 2021 European Quality of Government Index. You can read more about the survey and download the data here

Cost of 1gb mobile data in every country

r/dataisbeautiful - Cost of 1gb mobile data in every country

r/dataisbeautiful - Cost of 1gb mobile data in every country

Dataset: Visual Capitalist

Frequency of all digrams in 18 languages, diacritics included 

r/dataisbeautiful - Langues germaniques

Dataset (according to author): Dictionaries are scattered on the internet and had to be borrowed from several sources: the Scrabble3d project, and Linux spellcheck dictionaries. The data can be found in the folder “Avec_diacritiques”.

Criteria for choosing a dictionary:
– No proper nouns
– “Official” source if available
– Inclusion of inflected forms
– Among two lists, the largest was fancied
– No or very rare abbreviations if possible- but hard to detect in unknown languages and across hundreds of thousands of words.

Mapped: The World’s Nuclear Reactor Landscape

r/dataisbeautiful - Mapped: The World’s Nuclear Reactor Landscape

Dataset: Visual Capitalist

Database of 999 chemicals based on liver-specific carcinogenicity

The author found this dataset in a more accessible format upon searching for the keyword “CDPB” (Carcinogenic Potency Database) in the National Library of Medicine Catalog. Check out this parent website for the data source and dataset description. The dataset referenced in OP’s post concerns liver specific carcinogens, which are marked by the “liv” keyword as described in the dataset description’s Tissue Codes section.

SMS Spam Collection Data Set

DownloadData FolderData Set Description

The SMS Spam Collection is a public set of SMS labeled messages that have been collected for mobile phone spam research

Open Datasets for Autonomous Driving

A2D2 DatasetApolloScape Dataset Argoverse Dataset Berkeley DeepDrive Dataset

CityScapes DatasetComma2k19 DatasetGoogle-Landmarks Dataset

KITTI Vision Benchmark SuiteLeddarTech PixSet Dataset Level 5 Open DatanuScenes Dataset

Oxford Radar RobotCar DatasetPandaSet Udacity Self Driving Car Dataset Waymo Open Dataset

Open Dataset people are looking for [Help if you can]

  1. Looking for Dataset on the outcomes of abstinence-only sex education.
  2. Funny Datasets for School Data Science Project [1, 2, 3, 4, 5]
  3. Need a dataset for English practicing chatbot. [1, 2 ]
  4. Creating a dataset for plant disease recognition [1, 2 ]
  5. Central Bank Speeches Dataset (Text data from 1997 to 2020 from 118 institutions) [1, 2]
  6. Cat Meow Classification dataset [1, 2]
  7. Looking for Raw Data: Camping / Outdoors Travel in United States trends, etc [1, 2 ]
  8. Looking for Data set of horse race results / lottery results any results related to gambling [1, 2, 3]
  9. Looking for Football (Soccer) Penalties Dataset [1, 2]
  10. Looking for public datasets on baseball [1, 2, 3]
  11. Looking for Datasets on edge computing for AI bandwidth usage, latency, memory, CPU/GPU resource usage? [1 ,2 ]
  12. Data set of people who died by suicide [1, 2 ]
  13. Supreme Court dataset with opinion text? [1, 2, 3, 4, https://storage.googleapis.com/scotus-db/scotus-db.db5]
  14. Dataset of employee attrition or turnover rate? [1, 2]
  15. Is there a Dataset for homophobic tweets? [1 ,2, 3, 4, ]
  16. Looking for a Machine condition Monitoring Dataset [1,2]
  17. Where to find data for credit risk analysis? [1, 2]
  18. Datasets on homicides anywhere in the world [1, 2]
  19. Looking for a dataset containing coronavirus self-test (if this is a thing globally) pictures for ML use
  20. Is there any transportation dataset with daily frequency? [1, 2]
  21. A Dataset of film Locations [1, 2 ]
  22. Looking for a classification dataset [1, 2, 3, 4, 5]
  23. Where can I search for macroeconomics data? [1, 2, 3, 4, 5, 6, 7]
  24. Looking for Beam alignment 5G vehicular networks dataset
  25. Looking for tidy dataset for multivariate analysis (PCA, FA, canonical correlations, clustering)
  26. Indian all types of Fuel location datasets [1, 2,]
  27. Spotify Playlists Dataset [1, 2]
  28. World News Headline Dataset. With Sentiment Scores. Free download in JSON format. Updated often. [1, 2]
  29. Are there any free open source recipe datasets for commercial use [1, 2, 3, 4, 5]
  30. Curated social network datasets with summary statistics and background info
  31. Looking for textile crop disease datasets such as jute, flax, hemp
  32. Shopify App Store and Chrome Webstore Datasets
  33. Looking for dataset for university chatbot
  34. Collecting real life (dirty/ugly) datasets for data analysis
  35. In Need of Food Additive/Ingredient Definition Database
  36. Recent smart phone sensor Dataset – Android
  37. Cracked Mobile Screen Image Dataset for Detection
  38. Looking for Chiller fault data in a chiller plant
  39. Looking for dataset that contains the genetic sequences of native plasmids?
  40. Looking for a dataset containing fetus size measurements at various gestational ages.
  41. Looking for datasets about mental health since 2021
  42. Do you know where to find a dataset with Graphical User Interfaces defects of web applications? [1, 2, 3 ]
  43. Looking for most popular accounts on social medias like Twitter, Tik Tok, instagram, [1, 2, 3]
  44. GPS dataset of grocery stores
  45. What is the easiest way to bulk download all of the data from this epidemiology website? (~20,000 files)
  46. Looking for Dataset on Percentage of death by US state and Canadian province grouped by cause of death?
  47. Looking for Social engineering attack dataset in social media
  48. Steam Store Games (Clean dataset) by Nik Davis
  49. Dataset that lists all US major hospitals by county
  50. Another Data that list all US major hospitals by county
  51. Looking for open source data relating privacy behavior or related marketing sets about the trustworthiness of responders?
  52. Looking for a dataset that tracks median household income by country and year
  53. Dataset on the number of specific surgical procedures performed in the US (yearly)
  54. Looking for a dataset from reddit or twitter on top posts or tweets related to crypto currency
  55. Looking for Image and flora Dataset of All Known Plants, Trees and Shrubs
  56. US total fertility rates data one the state level
  57. Dataset of Net Worth of *World* Politicians
  58. Looking for water wells and borehole datasets