What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions

AWS Certified Cloud Practitioner Exam Preparation

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions

AWS certifications are becoming increasingly popular as the demand for AWS-skilled workers continues to grow. AWS certifications show that an individual has the necessary skills to work with AWS technologies, which can be beneficial for both job seekers and employers. AWS-certified individuals can often command higher salaries and are more likely to be hired for AWS-related positions. So, what are the top 10 AWS jobs that you can get with an AWS certification?

1. AWS Solutions Architect / Cloud Architect:

AWS solutions architects are responsible for designing, implementing, and managing AWS solutions. They work closely with other teams to ensure that AWS solutions are designed and implemented correctly.

AWS Architects, AWS Cloud Architects, and AWS solutions architects spend their time architecting, building, and maintaining highly available, cost-efficient, and scalable AWS cloud environments. They also make recommendations regarding AWS toolsets and keep up with the latest in cloud computing.

Professional AWS cloud architects deliver technical architectures and lead implementation efforts, ensuring new technologies are successfully integrated into customer environments. This role works directly with customers and engineers, providing both technical leadership and an interface with client-side stakeholders.

What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions
AWS SAA-C02 SAA-C03 Exam Prep

Average yearly salary: $148,000-$158,000 USD


2. AWS SysOps Administrator / Cloud System Administrators:

AWS sysops administrators are responsible for managing and operating AWS systems. They work closely with AWS developers to ensure that systems are running smoothly and efficiently.

A Cloud Systems Administrator, or AWS SysOps administrator, is responsible for the effective provisioning, installation/configuration, operation, and maintenance of virtual systems, software, and related infrastructures. They also maintain analytics software and build dashboards for reporting.

Average yearly salary: $97,000-$107,000 USD


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

3. AWS DevOps Engineer:

AWS devops engineers are responsible for designing and implementing automated processes for Amazon Web Services. They work closely with other teams to ensure that processes are efficient and effective.

AWS DevOps engineers design AWS cloud solutions that impact and improve the business. They also perform server maintenance and implement any debugging or patching that may be necessary. Among other DevOps things!

Average yearly salary: $118,000-$138,000 USD

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions
AWS Developer Associate DVA-C01 Exam Prep

4. AWS Cloud Engineer:

AWS cloud engineers are responsible for designing, implementing, and managing cloud-based solutions using AWS technologies. They work closely with other teams to ensure that solutions are designed and implemented correctly.

5. AWS Network Engineer:

AWS network engineers are responsible for designing, implementing, and managing networking solutions using AWS technologies. They work closely with other teams to ensure that networking solutions are designed and implemented correctly.

Cloud network specialists, engineers, and architects help organizations successfully design, build, and maintain cloud-native and hybrid networking infrastructures, including integrating existing networks with AWS cloud resources.

Average yearly salary: $107,000-$127,000 USD

6. AWS Security Engineer:

AWS security engineers are responsible for ensuring the security of Amazon Web Services environments. They work closely with other teams to identify security risks and implement controls to mitigate those risks.

Cloud security engineers provide security for AWS systems, protect sensitive and confidential data, and ensure regulatory compliance by designing and implementing security controls according to the latest security best practices.

Average yearly salary: $132,000-$152,000 USD

What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions
AWS Certified Security Specialty

7. AWS Database administrator:

As a database administrator on Amazon Web Services (AWS), you’ll be responsible for setting up, maintaining, and securing databases hosted on the Amazon cloud platform. You’ll work closely with other teams to ensure that databases are properly configured and secured.

8. Cloud Support Engineer:

Support engineers are responsible for providing technical support to AWS customers. They work closely with customers to troubleshoot problems and provide resolution within agreed upon SLAs.

9. Sales Engineer:

Sales engineers are responsible for working with sales teams to generate new business opportunities through the use of AWS products and services .They must have a deep understanding of AWS products and how they can be used by potential customers to solve their business problems .

10. Cloud Developer

An AWS Developer builds software services and enterprise-level applications. Generally, previous experience working as a software developer and a working knowledge of the most common cloud orchestration tools is required to get and succeed at an AWS cloud developer job

Djamgatech: Build the skills that’ll drive your career into six figures: Get Djamgatech.

Average yearly salary: $132,000 USD

11. Cloud Consultant

Cloud consultants provide organizations with technical expertise and strategy in designing and deploying AWS cloud solutions or in consulting on specific issues such as performance, security, or data migration.

Average yearly salary: $104,000-$124,000

12. Cloud Data Architect

Cloud data architects and data engineers may be cloud database administrators or data analytics professionals who know how to leverage AWS database resources, technologies, and services to unlock the value of enterprise data.

Average yearly salary: $130,000-$140,000 USD

What are the Top 10 AWS jobs you can get with an AWS certification in 2022 plus AWS Interview Questions
AWS Data analytics DAS-C01 Exam Prep

Getting a job after getting an AWS certification

The field of cloud computing will continue to grow and even more different types of jobs will surface in the future.

AWS certified professionals are in high demand across a variety of industries. AWS certs can open the door to a number of AWS jobs, including cloud engineer, solutions architect, and DevOps engineer.

Through studying and practice, any of the listed jobs could becoming available to you if you pass your AWS certification exams. Educating yourself on AWS concepts plays a key role in furthering your career and receiving not only a higher salary, but a more engaging position.

Source: 8 AWS jobs you can get with an AWS certification

AWS Tech Jobs  Interview Questions in 2022

Graphs

1) Process Ordering – LeetCode link…

Ace the Microsoft Azure Fundamentals AZ-900 Certification Exam: Pass the Azure Fundamentals Exam with Ease

2) Number of Islands – LeetCode link…

3) k Jumps on Grid – Loading…)

Sort

1) Finding Prefix in Dictionary – LeetCode Link…

Tree

1) Binary Tree Top Down View – LeetCode link…

2) Traversing binary tree in an outward manner.

3) Diameter of a binary tree [Path is needed] – Diameter of a Binary Tree – GeeksforGeeks

Sliding window

1) Contains Duplicates III – LeetCode link…

2) Minimum Window Substring [Variation of this question] – LeetCode link..

Linked List

1) Reverse a Linked List II – LeetCode link…

2) Remove Loop From Linked List – Remove Loop in Linked List

3) Reverse a Linked List in k-groups – LeetCode link…

Binary Search

1) Search In rotate sorted Array – LeetCode link…

Solution:

def pivotedBinarySearch(arr, n, key):
 
    pivot = findPivot(arr, 0, n-1)
 
    # If we didn't find a pivot,
    # then array is not rotated at all
    if pivot == -1:
        return binarySearch(arr, 0, n-1, key)
 
    # If we found a pivot, then first
    # compare with pivot and then
    # search in two subarrays around pivot
    if arr[pivot] == key:
        return pivot
    if arr[0] <= key:
        return binarySearch(arr, 0, pivot-1, key)
    return binarySearch(arr, pivot + 1, n-1, key)
 
 
# Function to get pivot. For array
# 3, 4, 5, 6, 1, 2 it returns 3
# (index of 6)
def findPivot(arr, low, high):
 
    # base cases
    if high < low:
        return -1
    if high == low:
        return low
 
    # low + (high - low)/2;
    mid = int((low + high)/2)
 
    if mid < high and arr[mid] > arr[mid + 1]:
        return mid
    if mid > low and arr[mid] < arr[mid - 1]:
        return (mid-1)
    if arr[low] >= arr[mid]:
        return findPivot(arr, low, mid-1)
    return findPivot(arr, mid + 1, high)
 
# Standard Binary Search function
def binarySearch(arr, low, high, key):
 
    if high < low:
        return -1
 
    # low + (high - low)/2;
    mid = int((low + high)/2)
 
    if key == arr[mid]:
        return mid
    if key > arr[mid]:
        return binarySearch(arr, (mid + 1), high,
                            key)
    return binarySearch(arr, low, (mid - 1), key)
 
# Driver program to check above functions
# Let us search 3 in below array
if __name__ == '__main__':
    arr1 = [5, 6, 7, 8, 9, 10, 1, 2, 3]
    n = len(arr1)
    key = 3
    print("Index of the element is : ", \
          pivotedBinarySearch(arr1, n, key))
 
# This is contributed by Smitha Dinesh Semwal

Arrays

1) Max bandWidth [Priority Queue, Sorting] – Loading…

2) Next permutation – Loading…

3) Largest Rectangle in Histogram – Loading…

Content by – Sandeep Kumar

#AWS #interviews #leetcode #questions #array #sorting #queue #loop #tree #graphs #amazon #sde —-#interviewpreparation #coding #computerscience #softwareengineer

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

What are the Top 5 things that can say a lot about a software engineer or programmer’s quality?

When it comes to the quality of a software engineer or programmer, there are a few key things that can give you a good indication. First, take a look at their code quality. A good software engineer will take pride in their work and produce clean, well-organized code. They will also be able to explain their code concisely and confidently. Another thing to look for is whether they are up-to-date on the latest coding technologies and trends. A good programmer will always be learning and keeping up with the latest industry developments. Finally, pay attention to how they handle difficult problems. A good software engineer will be able to think creatively and come up with innovative solutions to complex issues. If you see these qualities in a software engineer or programmer, chances are they are of high quality.

Below are the top 5 things can say a lot about a software engineer/ programmer’s quality?

  1. The number of possible paths through the code (branch points) is minimized. Top quality code tends to be much more straight line than poor code. As a result, the author can design, code and test very quickly and is often looked at as a programming guru. In addition this code is far more resilient in Production.
  2. The code clearly matches the underlying business requirements and can therefore be understood very quickly by new resources. As a result there is much less tendency for a maintenance programmer to break the basic design as opposed to spaghetti code where small changes can have catastrophic effects.
  3. There is an overall sense of pride in the source code itself. If the enterprise has clear written standards, these are followed to the letter. If not, the code is internally consistent in terms of procedure/object, function/method or variable/attribute naming. Also indentation and continuations are universally consistent throughout. Last but not least, the majority of code blocks are self-evident to the requirements and where not the case, adequate purpose focused documentation is provided.

    In general, I have seen two types of programs provided for initial Production deployment. One looks like it was just written moments ago and the other looks like it has had 20 years of maintenance performed on it. Unfortunately, the authors of the second type cannot generally see the difference so it is a lost cause and we just have to continue to deal with the problems.
  4. In today’s programming environment, a project may span many platforms, languages etc. A simple web page may invoke an API which in turn accesses a database. For this example lets say JavaScript – Rest API – C# – SQL – RDBMS. The programmer can basically embed logic anywhere in this chain, but needs to be aware of reuse, performance and maintenance issues. For instance, if a part of the process requires access to three database tables, it is both faster and clearer to allow the DBMS engine return a single query than compare the tables in the API code. Similarly every business rule coded in the client side reduces re-usability potential.
    Top quality developers understand these issues and can optimize their designs to take advantages of the strengths of the component technologies.
  5. The ability to stay current with new trends and technologies. Technology is constantly evolving, and a good software engineer or programmer should be able to stay up-to-date on the latest trends and technologies in order to be able to create the best possible products.

To conclude:

Below are other things to consider when hiring good software engineers or programmers:


  1. The ability to write clean, well-organized code. This is a key indicator of a good software engineer or programmer. The ability to write code that is easy to read and understand is essential for creating high-quality software.
  2. The ability to test and debug code. A good coder should be able to test their code thoroughly and identify and fix any errors that may exist.
  3. The ability to write efficient code. Software engineering is all about creating efficient solutions to problems. A good software engineer or programmer will be able to write code that is efficient and effective.
  4. The ability to work well with others. Software engineering is typically a team-based effort. A good software engineer or programmer should be able to work well with others in order to create the best possible product.
  5. The ability to stay current with new trends and technologies.

Why is there a lack of diversity among software engineers in tech companies in USA and Canada?

What are the top 10 biggest lessons you have learned from the corporate world?

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Why is there a lack of diversity among software engineers in tech companies in USA and Canada?

One of the most significant problems facing the tech industry is a lack of diversity among software engineers. In the United States and Canada, women and visible minorities are grossly underrepresented in the field. This problem is often attributed to the pipelines that feed into the industry. For example, women are less likely than men to study computer science in college. However, this does not fully explain the disparity. Hiring practices at tech companies are also to blame. Studies have shown that companies are biased against female and minority candidates. They are more likely to hire white men with similar educational backgrounds and work experience. This lack of diversity has negative consequences for both the individuals affected and the companies themselves. Immersive technologies, such as virtual reality, are being designed and developed primarily by white men. This creates a product that does not reflect the needs or perspectives of a diverse population. In addition, companies with diverse teams have been shown to perform better than those without diversity. They are more innovative and adaptable to change. The lack of diversity among software engineers is a problem that needs to be addressed by both educators and tech companies if the industry is to thrive in the future.

Why is there a lack of diversity among software engineers in tech companies in USA and Canada?
FAANGM Compensations

A prominent Engineer from Silicon Valley said  this about the topic: Women and minorities are systematically discouraged from taking STEM subjects starting around the third grade. Fewer of them excel in math and science in high school. Fewer go to college. Women and minorities don’t see women and minorities in tech roles in the media. Some hiring managers are racist, misogynist scumbags. Some co-workers are dismissive of them. They have trouble finding mentors. The women, at least, are encouraged continuously to drop out and make fat babies.

This all forms a pretty effective filter.  By Kurt Guntheroth.

Find Local and Remote Real Time Jobs at https://inRealtimeJobs.com


Software industry has mostly adopted and promoted conformist culture, thanks to its leaders who mostly have been power mongers and Machiavellian by approach, with few exceptions here and there in few organizations. Conforming culture will inherently repel diversity since one is more assured of conforming staff in a known culture rather than more diverse culture. Most of these so called pseudo leaders don’t really seek diverse ideas which inherently stem from diverse cultures. And so such organizations never end up becoming diverse.

There aren’t enough women in the industry — in individual contributor roles and in leadership roles. There also aren’t enough African-Americans and Latinos.

Why does this matter? Many reasons. One big one is that exclusion leads to more exclusion. When one gender/ethnic group is significantly underrepresented in a workforce, strong biases bake in -> the people in the affected group think they don’t belong at the compan(ies) and the people at the companies have an insular bias to pick people from their networks/people who share their backgrounds.


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

Silicon Valley is the hottest growth sector in the US and will continue to create the best career and wealth opportunities over the next 20–30 years. It’s really not good that the industry isn’t absorbing more women, African-Americans, and Latinos.

To conclude:

Technology is a rapidly growing industry with a huge demand for qualified software engineers. However, there is a lack of diversity among software engineers in tech companies in the USA and Canada. Women and minorities are greatly underrepresented in the field of software engineering. This is due to several factors, including the misogynistic and racist culture of many tech companies. The lack of diversity among software engineers has a negative impact on the quality of products and services offered by tech companies. It also limits the ability of these companies to innovate and serve a wide range of customers. In order to increase diversity among software engineers, tech companies need to change their hiring practices and create an inclusive environment that values all types of people.

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

Find local and Remote Real Time Jobs at https://inRealtimeJobs.com

Examining the Fragmented Data on Black Entrepreneurship in North America

Financing Black Businesses in Canada and USA: Challenges and Opportunities

What are popular hobbies among Software Engineers?

Software Engineers Hobbies

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

What are popular hobbies among Software Engineers?

Just like any other profession, Software Engineers have their own unique set of hobbies that they enjoy. While some programmers prefer to relax by playing video games or watching TV, others like to get more involved in their hobbies and spend their free time learning new things or being creative. If you’re curious about what some of the most popular hobbies among software engineers are, read on!

1. Reading
2. Playing video games
3. Working on personal projects
4. Cooking
5. Hiking/running
6. Watching TV/movies

 

Stereotypically, video games and board games are supposed to be popular, but of the many programmers I know, I can think of only two that are really into games enough to call it a hobby rather than (as they are for most of us) a casual diversion. One is more board-gamery, and the other is more videogamer-uh-erery.


What are popular hobbies among Software Engineers?
What are the Top 5 things that can say a lot about a software engineer or programmer’s quality?

Outside of that, hobbies I know of amongst programmers I know (and I include myself) are:

  • Photography

  • Playing musical instruments and/or music production

  • Hiking

  • Bicycling

  • Motorcycling

  • Cars (motorsport, restoration, etc.)

  • 4x4s and off-roading

  • Electronics

  • Retrocomputing

  • Software side projects

  • Pets, usually dogs & cats but sometimes exotics

  • Do-it-yourself plumbing/electrical/carpentry/plastering/concrete

  • Cooking and/or foodie pursuits like visiting restaurants or inventing cocktails

  • Coffee (machines, roasting, blending, drinking)

  • Lego™

  • Card games (often poker or blackjack)

  • Rock climbing

  • Fishing

  • Target shooting

  • Archery

  • Kayaking

  • Painting, drawing or sculpture

  • Flying model airplanes, helicopters and/or rockets

  • Golf, football, tennis, raquetball, squash (spectatorship and/or playing)

  • Skiing

  • Chess

  • Soccer/Football

  • Martial Arts

  • Blogging

Summary:

How do Software Engineers spend their free time? What are their hobbies? Well, according to a recent survey, the most popular hobby among Software Engineers is programming. But that’s not all – they also enjoy playing video games, reading books, and spending time with friends and family. So what does this tell us about Software Engineers? Well, for one, they’re passionate about technology and love to stay up-to-date on the latest innovations. But they’re also well-rounded individuals who enjoy a variety of activities outside of work. So if you’re looking for someone to chat with about the latest tech trends or just want to find someone to play video games with, then a Software Engineer is your best bet!


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

Source: What are popular hobbies among Software Engineers on Quora

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

Google interview questions for various roles and How to Ace the Google Software Engineering Interview?

Google Interview Questions and Answers

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Google interview questions for various roles and How to Ace the Google Software Engineering Interview?

Google is one of the most sought-after employers in the world, known for their cutting-edge technology and innovative products.

If you’re lucky enough to land an interview with Google, you can expect to be asked some challenging questions. Google is known for their brainteasers and algorithmic questions, so it’s important to brush up on your coding skills before the interview. However, Google also values creativity and out-of-the-box thinking, so don’t be afraid to think outside the box when answering questions. product managers need to be able to think strategically about Google’s products, while software engineers will need to demonstrate their technical expertise. No matter what role you’re interviewing for, remember to stay calm and confident, and you’ll be sure to ace the Google interview.

The interview process is notoriously difficult, with contenders being put through their paces with brain-teasers, algorithm questions, and intense coding challenges. However, Google interviews aren’t just designed to trip you up – they’re also an opportunity to show off your skills and demonstrate why you’re the perfect fit for the role. If you’re hoping to secure a Google career, preparation is key. Here are some top tips for acing the Google interview, whatever position you’re applying for.

Firstly, take some time to familiarize yourself with Google’s products and services. Google is such a huge company that it can be easy to get overwhelmed, but it’s important to remember that they started out as a search engine. Having a solid understanding of how Google works will give you a good foundation to build upon during the interview process. Secondly, practice your coding skills. Google interviews are notoriously difficult, and many contenders fail at the first hurdle because they’re not prepared for the level of difficulty.

The company is known for its rigorous interview process, which often includes a mix of coding, algorithm, and behavioral questions. While Google interview questions can vary depending on the role, there are some common themes that arise. For software engineering positions, candidates can expect to be asked questions about their coding skills and experience. For product manager roles, Google interviewers often focus on behavioral questions, such as how the candidate has handled difficult decisions in the past. Quantitative compensation analyst candidates may be asked math-based questions, while AdWords Associates may be asked about Google’s advertising products and policies. Google is known for being an intense place to work, so it’s important for interviewees to go into the process prepared and ready to impress. Ultimately, nailing the Google interview isn’t just about having the right answers – it’s also about having the right attitude.


Below are some of the questions asked during Google Interview for various roles:

Google interview questions for various roles and How to Ace the Google Software Engineering Interview?
Google interview questions for various roles and How to Ace the Google Software Engineering Interview?

 


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

Google Interview Questions: Product Marketing Manager

  • Why do you want to join Google?
  • What do you know about Google’s product and technology?
  • If you are Product Manager for Google’s Adwords, how do you plan to market this?
  • What would you say during an AdWords or AdSense product seminar?
  • Who are Google’s competitors, and how does Google compete with them?
  • Have you ever used Google’s products? Gmail?
  • What’s a creative way of marketing Google’s brand name and product?
  • If you are the product marketing manager for Google’s Gmail product, how do you plan to market it so as to achieve 100 million customers in 6 months?
  • How much money you think Google makes daily from Gmail ads?
  • Name a piece of technology you’ve read about recently. Now tell me your own creative execution for an ad for that product.
  • Say an advertiser makes $0.10 every time someone clicks on their ad. Only 20% of people who visit the site click on their ad. How many people need to visit the site for the advertiser to make $20?
  • Estimate the number of students who are college seniors, attend four-year schools, and graduate with a job in the United States every year.

Google Interview Questions: Product Manager

  • How would you boost the GMail subscription base?
  • What is the most efficient way to sort a million integers?
  • How would you re-position Google’s offerings to counteract competitive threats from Microsoft?
  • How many golf balls can fit in a school bus?
  • You are shrunk to the height of a nickel and your mass is proportionally reduced so as to maintain your original density. You are then thrown into an empty glass blender. The blades will start moving in 60 seconds. What do you do?
  • How much should you charge to wash all the windows in Seattle?
  • How would you find out if a machine’s stack grows up or down in memory?
  • Explain a database in three sentences to your eight-year-old nephew.
  • How many times a day does a clock’s hands overlap?
  • You have to get from point A to point B. You don’t know if you can get there. What would you do?
  • Imagine you have a closet full of shirts. It’s very hard to find a shirt. So what can you do to organize your shirts for easy retrieval?
  • Every man in a village of 100 married couples has cheated on his wife. Every wife in the village instantly knows when a man other than her husband has cheated, but does not know when her own husband has. The village has a law that does not allow for adultery. Any wife who can prove that her husband is unfaithful must kill him that very day. The women of the village would never disobey this law. One day, the queen of the village visits and announces that at least one husband has been unfaithful. What happens?
  • In a country in which people only want boys, every family continues to have children until they have a boy. If they have a girl, they have another child. If they have a boy, they stop. What is the proportion of boys to girls in the country?
  • If the probability of observing a car in 30 minutes on a highway is 0.95, what is the probability of observing a car in 10 minutes (assuming constant default probability)?
  • If you look at a clock and the time is 3:15, what is the angle between the hour and the minute hands? (The answer to this is not zero!)
  • Four people need to cross a rickety rope bridge to get back to their camp at night. Unfortunately, they only have one flashlight and it only has enough light left for seventeen minutes. The bridge is too dangerous to cross without a flashlight, and it’s only strong enough to support two people at any given time. Each of the campers walks at a different speed. One can cross the bridge in 1 minute, another in 2 minutes, the third in 5 minutes, and the slow poke takes 10 minutes to cross. How do the campers make it across in 17 minutes?
  • You are at a party with a friend and 10 people are present including you and the friend. your friend makes you a wager that for every person you find that has the same birthday as you, you get $1; for every person he finds that does not have the same birthday as you, he gets $2. would you accept the wager?
  • How many piano tuners are there in the entire world?
  • You have eight balls all of the same size. 7 of them weigh the same, and one of them weighs slightly more. How can you find the ball that is heavier by using a balance and only two weighings?
  • You have five pirates, ranked from 5 to 1 in descending order. The top pirate has the right to propose how 100 gold coins should be divided among them. But the others get to vote on his plan, and if fewer than half agree with him, he gets killed. How should he allocate the gold in order to maximize his share but live to enjoy it? (Hint: One pirate ends up with 98 percent of the gold.)
  • You are given 2 eggs. You have access to a 100-story building. Eggs can be very hard or very fragile means it may break if dropped from the first floor or may not even break if dropped from 100th floor. Both eggs are identical. You need to figure out the highest floor of a 100-story building an egg can be dropped without breaking. The question is how many drops you need to make. You are allowed to break 2 eggs in the process.
  • Describe a technical problem you had and how you solved it.
  • How would you design a simple search engine?
  • Design an evacuation plan for San Francisco.
  • There’s a latency problem in South Africa. Diagnose it.
  • What are three long term challenges facing Google?
  • Name three non-Google websites that you visit often and like. What do you like about the user interface and design? Choose one of the three sites and comment on what new feature or project you would work on. How would you design it?
  • If there is only one elevator in the building, how would you change the design? How about if there are only two elevators in the building?
  • How many vacuum’s are made per year in USA?

Google Interview Questions: Software Engineer

  • Why are manhole covers round?
  • What is the difference between a mutex and a semaphore? Which one would you use to protect access to an increment operation?
  • A man pushed his car to a hotel and lost his fortune. What happened?
  • Explain the significance of “dead beef”.
  • Write a C program which measures the the speed of a context switch on a UNIX/Linux system.
  • Given a function which produces a random integer in the range 1 to 5, write a function which produces a random integer in the range 1 to 7.
  • Describe the algorithm for a depth-first graph traversal.
  • Design a class library for writing card games.
  • You need to check that your friend, Bob, has your correct phone number, but you cannot ask him directly. You must write a the question on a card which and give it to Eve who will take the card to Bob and return the answer to you. What must you write on the card, besides the question, to ensure Bob can encode the message so that Eve cannot read your phone number?
  • How are cookies passed in the HTTP protocol?
  • Design the SQL database tables for a car rental database.
  • Write a regular expression which matches a email address.
  • Write a function f(a, b) which takes two character string arguments and returns a string containing only the characters found in both strings in the order of a. Write a version which is order N-squared and one which is order N.
  • You are given a the source to a application which is crashing when run. After running it 10 times in a debugger, you find it never crashes in the same place. The application is single threaded, and uses only the C standard library. What programming errors could be causing this crash? How would you test each one?
  • Explain how congestion control works in the TCP protocol.
  • In Java, what is the difference between final, finally, and finalize?
  • What is multithreaded programming? What is a deadlock?
  • Write a function (with helper functions if needed) called to Excel that takes an excel column value (A,B,C,D…AA,AB,AC,… AAA..) and returns a corresponding integer value (A=1,B=2,… AA=26..).
  • You have a stream of infinite queries (ie: real time Google search queries that people are entering). Describe how you would go about finding a good estimate of 1000 samples from this never ending set of data and then write code for it.
  • Tree search algorithms. Write BFS and DFS code, explain run time and space requirements. Modify the code to handle trees with weighted edges and loops with BFS and DFS, make the code print out path to goal state.
  • You are given a list of numbers. When you reach the end of the list you will come back to the beginning of the list (a circular list). Write the most efficient algorithm to find the minimum # in this list. Find any given # in the list. The numbers in the list are always increasing but you don’t know where the circular list begins, ie: 38, 40, 55, 89, 6, 13, 20, 23, 36.
  • Describe the data structure that is used to manage memory. (stack)
  • What’s the difference between local and global variables?
  • If you have 1 million integers, how would you sort them efficiently? (modify a specific sorting algorithm to solve this)
  • In Java, what is the difference between static, final, and const. (if you don’t know Java they will ask something similar for C or C++).
  • Talk about your class projects or work projects (pick something easy)… then describe how you could make them more efficient (in terms of algorithms).
  • Suppose you have an NxN matrix of positive and negative integers. Write some code that finds the sub-matrix with the maximum sum of its elements.
  • Write some code to reverse a string.
  • Implement division (without using the divide operator, obviously).
  • Write some code to find all permutations of the letters in a particular string.
  • What method would you use to look up a word in a dictionary?
  • Imagine you have a closet full of shirts. It’s very hard to find a shirt. So what can you do to organize your shirts for easy retrieval?
  • You have eight balls all of the same size. 7 of them weigh the same, and one of them weighs slightly more. How can you fine the ball that is heavier by using a balance and only two weighings?
  • What is the C-language command for opening a connection with a foreign host over the internet?
  • Design and describe a system/application that will most efficiently produce a report of the top 1 million Google search requests. These are the particulars: 1) You are given 12 servers to work with. They are all dual-processor machines with 4Gb of RAM, 4x400GB hard drives and networked together.(Basically, nothing more than high-end PC’s) 2) The log data has already been cleaned for you. It consists of 100 Billion log lines, broken down into 12 320 GB files of 40-byte search terms per line. 3) You can use only custom written applications or available free open-source software.
  • There is an array A[N] of N numbers. You have to compose an array Output[N] such that Output[i] will be equal to multiplication of all the elements of A[N] except A[i]. For example Output[0] will be multiplication of A[1] to A[N-1] and Output[1] will be multiplication of A[0] and from A[2] to A[N-1]. Solve it without division operator and in O(n).
  • There is a linked list of numbers of length N. N is very large and you don’t know N. You have to write a function that will return k random numbers from the list. Numbers should be completely random. Hint: 1. Use random function rand() (returns a number between 0 and 1) and irand() (return either 0 or 1) 2. It should be done in O(n).
  • Find or determine non existence of a number in a sorted list of N numbers where the numbers range over M, M>> N and N large enough to span multiple disks. Algorithm to beat O(log n) bonus points for constant time algorithm.
  • You are given a game of Tic Tac Toe. You have to write a function in which you pass the whole game and name of a player. The function will return whether the player has won the game or not. First you to decide which data structure you will use for the game. You need to tell the algorithm first and then need to write the code. Note: Some position may be blank in the game। So your data structure should consider this condition also.
  • You are given an array [a1 To an] and we have to construct another array [b1 To bn] where bi = a1*a2*…*an/ai. you are allowed to use only constant space and the time complexity is O(n). No divisions are allowed.
  • How do you put a Binary Search Tree in an array in a efficient manner. Hint :: If the node is stored at the ith position and its children are at 2i and 2i+1(I mean level order wise)Its not the most efficient way.
  • How do you find out the fifth maximum element in an Binary Search Tree in efficient manner. Note: You should not use use any extra space. i.e sorting Binary Search Tree and storing the results in an array and listing out the fifth element.
  • Given a Data Structure having first n integers and next n chars. A = i1 i2 i3 … iN c1 c2 c3 … cN.Write an in-place algorithm to rearrange the elements of the array ass A = i1 c1 i2 c2 … in cn
  • Given two sequences of items, find the items whose absolute number increases or decreases the most when comparing one sequence with the other by reading the sequence only once.
  • Given That One of the strings is very very long , and the other one could be of various sizes. Windowing will result in O(N+M) solution but could it be better? May be NlogM or even better?
  • How many lines can be drawn in a 2D plane such that they are equidistant from 3 non-collinear points?
  • Let’s say you have to construct Google maps from scratch and guide a person standing on Gateway of India (Mumbai) to India Gate(Delhi). How do you do the same?
  • Given that you have one string of length N and M small strings of length L. How do you efficiently find the occurrence of each small string in the larger one?
  • Given a binary tree, programmatically you need to prove it is a binary search tree.
  • You are given a small sorted list of numbers, and a very very long sorted list of numbers – so long that it had to be put on a disk in different blocks. How would you find those short list numbers in the bigger one?
  • Suppose you have given N companies, and we want to eventually merge them into one big company. How many ways are theres to merge?
  • Given a file of 4 billion 32-bit integers, how to find one that appears at least twice?
  • Write a program for displaying the ten most frequent words in a file such that your program should be efficient in all complexity measures.
  • Design a stack. We want to push, pop, and also, retrieve the minimum element in constant time.
  • Given a set of coin denominators, find the minimum number of coins to give a certain amount of change.
  • Given an array, i) find the longest continuous increasing subsequence. ii) find the longest increasing subsequence.
  • Suppose we have N companies, and we want to eventually merge them into one big company. How many ways are there to merge?
  • Write a function to find the middle node of a single link list.
  • Given two binary trees, write a compare function to check if they are equal or not. Being equal means that they have the same value and same structure.
  • Implement put/get methods of a fixed size cache with LRU replacement algorithm.
  • You are given with three sorted arrays ( in ascending order), you are required to find a triplet ( one element from each array) such that distance is minimum.
  • Distance is defined like this : If a[i], b[j] and c[k] are three elements then distance=max(abs(a[i]-b[j]),abs(a[i]-c[k]),abs(b[j]-c[k]))” Please give a solution in O(n) time complexity
  • How does C++ deal with constructors and deconstructors of a class and its child class?
  • Write a function that flips the bits inside a byte (either in C++ or Java). Write an algorithm that take a list of n words, and an integer m, and retrieves the mth most frequent word in that list.
  • What’s 2 to the power of 64?
  • Given that you have one string of length N and M small strings of length L. How do you efficiently find the occurrence of each small string in the larger one?
  • How do you find out the fifth maximum element in an Binary Search Tree in efficient manner.
  • Suppose we have N companies, and we want to eventually merge them into one big company. How many ways are there to merge?
  • There is linked list of millions of node and you do not know the length of it. Write a function which will return a random number from the list.
  • You need to check that your friend, Bob, has your correct phone number, but you cannot ask him directly. You must write a the question on a card which and give it to Eve who will take the card to Bob and return the answer to you. What must you write on the card, besides the question, to ensure Bob can encode the message so that Eve cannot read your phone number?
  • How long it would take to sort 1 trillion numbers? Come up with a good estimate.
  • Order the functions in order of their asymptotic performance: 1) 2^n 2) n^100 3) n! 4) n^n
  • There are some data represented by(x,y,z). Now we want to find the Kth least data. We say (x1, y1, z1) > (x2, y2, z2) when value(x1, y1, z1) > value(x2, y2, z2) where value(x,y,z) = (2^x)*(3^y)*(5^z). Now we can not get it by calculating value(x,y,z) or through other indirect calculations as lg(value(x,y,z)). How to solve it?
  • How many degrees are there in the angle between the hour and minute hands of a clock when the time is a quarter past three?
  • Given an array whose elements are sorted, return the index of a the first occurrence of a specific integer. Do this in sub-linear time. I.e. do not just go through each element searching for that element.
  • Given two linked lists, return the intersection of the two lists: i.e. return a list containing only the elements that occur in both of the input lists.
  • What’s the difference between a hashtable and a hashmap?
  • If a person dials a sequence of numbers on the telephone, what possible words/strings can be formed from the letters associated with those numbers?
  • How would you reverse the image on an n by n matrix where each pixel is represented by a bit?
  • Create a fast cached storage mechanism that, given a limitation on the amount of cache memory, will ensure that only the least recently used items are discarded when the cache memory is reached when inserting a new item. It supports 2 functions: String get(T t) and void put(String k, T t).
  • Create a cost model that allows Google to make purchasing decisions on to compare the cost of purchasing more RAM memory for their servers vs. buying more disk space.
  • Design an algorithm to play a game of Frogger and then code the solution. The object of the game is to direct a frog to avoid cars while crossing a busy road. You may represent a road lane via an array. Generalize the solution for an N-lane road.
  • What sort would you use if you had a large data set on disk and a small amount of ram to work with?
  • What sort would you use if you required tight max time bounds and wanted highly regular performance.
  • How would you store 1 million phone numbers?
  • Design a 2D dungeon crawling game. It must allow for various items in the maze – walls, objects, and computer-controlled characters. (The focus was on the class structures, and how to optimize the experience for the user as s/he travels through the dungeon.)
  • What is the size of the C structure below on a 32-bit system? On a 64-bit?

struct foo {

char a;
char* b;
};
  • A triomino is formed by joining three unit-sized squares in an L-shape. A mutilated chessboard is made up of 64 unit-sized squares arranged in an 8-by-8 square, minus the top left square.

    Design an algorithm which computes a placement of 21 triominos that covers the mutilated chessboard.2.

  • The mathematician G. H. Hardy was on his way to visit his collaborator S. Ramanujan who was in the hospital. Hardy remarked to Ramanujan that he traveled in a taxi cab with license plate 1729, which seemed a dull number. To this, Ramanujan replied that 1729 was a very interesting number – it was the smallest number expressible as the sum of cubes of two numbers in two different ways. Indeed, 10x10x10 + 9x9x9 = 12x12x12 + 1x1x1 = 1729.

    Given an arbitrary positive integer, how would you determine if it can be expressed as a sum of two cubes?

  • There are fifty coins in a line—these could be pennies, nickels, dimes, or quarters. Two players, $F$ and $S$, take turns at choosing one coin each—they can only choose from the two coins at the ends of the line. The game ends when all the coins have been picked up. The player whose coins have the higher total value wins. Each player must select a coin when it is his turn, so the game ends in fifty turns.

    If you want to ensure you do not lose, would you rather go first or second? Design an efficient algorithm for computing the maximum amount of money the first player can win.

  • You are given two sorted arrays. Design an efficient algorithm for computing the k-th smallest element in the union of the two arrays. (Keep in mind that the elements may be repeated.)

  • It’s literally about 10 lines of code, give or take. It’s at the heart of merge sort.

    Merge 2  sorted lists in C language
    Merge 2 sorted linked lists in C language

    Reference: Here

     

 

Google Interview: Software Engineer in Test

  • Efficiently implement 3 stacks in a single array.
  • Given an array of integers which is circularly sorted, how do you find a given integer.
  • Write a program to find depth of binary search tree without using recursion.
  • Find the maximum rectangle (in terms of area) under a histogram in linear time.
  • Most phones now have full keyboards. Before there there three letters mapped to a number button. Describe how you would go about implementing spelling and word suggestions as people type.
  • Describe recursive mergesort and its runtime. Write an iterative version in C++/Java/Python.
  • How would you determine if someone has won a game of tic-tac-toe on a board of any size?
  • Given an array of numbers, replace each number with the product of all the numbers in the array except the number itself *without* using division.
  • Create a cache with fast look up that only stores the N most recently accessed items.
  • How to design a search engine? If each document contains a set of keywords, and is associated with a numeric attribute, how to build indices?
  • Given two files that has list of words (one per line), write a program to show the intersection.
  • What kind of data structure would you use to index annagrams of words? e.g. if there exists the word “top” in the database, the query for “pot” should list that.

Google Interview: Quantitative Compensation Analyst

  • What is the yearly standard deviation of a stock given the monthly standard deviation?
  • How many resumes does Google receive each year for software engineering?
  • Anywhere in the world, where would you open up a new Google office and how would you figure out compensation for all the employees at this new office?
  • What is the probability of breaking a stick into 3 pieces and forming a triangle?

Google Interview: Engineering Manager

  • You’re the captain of a pirate ship, and your crew gets to vote on how the gold is divided up. If fewer than half of the pirates agree with you, you die. How do you recommend apportioning the gold in such a way that you get a good share of the booty, but still survive?

Google Interview: AdWords Associate

  • How would you work with an advertiser who was not seeing the benefits of the AdWords relationship due to poor conversions?
  • How would you deal with an angry or frustrated advertisers on the phone?

Sources

To conclude:

Google is one of the most sought-after employers in the tech industry. The company is known for its rigorous interview process, which often includes a mix of coding, algorithm, and behavioural questions. While Google interview questions can vary depending on the role, there are some common themes that arise. For software engineering positions, candidates can expect to be asked questions about their coding skills and experience. For product manager roles, Google interviewers often focus on behavioral questions, such as how the candidate has handled difficult decisions in the past. Quantitative compensation analyst candidates may be asked math-based questions, while AdWords Associates may be asked about Google’s advertising products and policies. Google is known for being an intense place to work, so it’s important for interviewees to go into the process prepared and ready to impress. Ultimately, nailing the Google interview isn’t just about having the right answers – it’s also about having the right attitude.

Simply put, no.

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

There’s no doubt that Cracking The Coding Interview (CTCI) is a great tool for honing your coding skills.

But in today’s competitive job landscape, you need a lot more than sharp coding skills to get hired by Google.

Think about it.

Google receives about 3 million job applications every year.

But it hires less than 1% of those people.

Most of those who get the job (if they’re software engineers, at least) spent weeks or months practicing problems in CTCI and LeetCode before their interview.

But so did the people who don’t get hired.

So if a mastery of coding problems isn’t whats set the winners apart from the losers, what is?

The soft skills.

Believe it or not, soft skills matter a lot, even as a software engineer.

Here are three soft skills Google looks for that CTCI won’t help you with.

Djamgatech: Build the skills that’ll drive your career into six figures: Get Djamgatech.

#1 LEADERSHIP

You’d be amazed how many candidates overlook the importance of leadership as they try to get hired by Google.

They forget that recruiters are not looking for their ability to be a strong junior engineer, but their ability to develop into a strong senior engineer.

Recruiters need to know that you have the empathy to lead a team, and that you’re willing to pull up your socks when things go awry.

If you can’t show that you’re a leader in your interview, it won’t matter how good your code is—you won’t be getting hired.

#2 COMMUNICATION & TEAMWORK

Teamwork and communication are two other skill sets you won’t gain from CTCI.

And just like leadership, you need to demonstrate these skills if you expect to get an offer from Google.

Why?

Because building the world’s best technology is a team sport, and if you want to thrive on Team Google, you need to prove yourself as a team player.

Ace the Microsoft Azure Fundamentals AZ-900 Certification Exam: Pass the Azure Fundamentals Exam with Ease

Don’t overlook this.

Google and the other FAANG companies regularly pass up skilled engineers because they don’t believe they’ll be strong members of the larger team.

#3 MASTERY OVER AMBIGUITY

Google recruiters often throw highly ambiguous problems at candidates just to see how they handle them.

So if you can’t walk the recruiter through your process for solving it, they’re going to move on to someone else.

The ambiguous problems I’m talking about are not like the ones you face in CTCI. They’re much more open-ended, and there truly are no right answers.

These are the sort of questions you need a guide to help you navigate through. That’s why you need more guidance than what CTCI provides if you want to give yourself the best chance at getting an offer.


If you just want to hone your coding skills, CTCI is a good place to start.

But if you’re serious about getting a job at Google, I recommend a more comprehensive course like Tech Interview Pro, which was designed by ex-Google and ex-Facebook software engineers to help you succeed in all areas of the job hunt, from building your resume all the way to salary negotiations.

Whatever you do, don’t overlook the importance of soft skills on your journey to getting hired. They’ll be what clinches your spot.

Good luck!

Source:

What are some ways we can use machine learning and artificial intelligence for algorithmic trading in the stock market?

How do we know that the Top 3 Voice Recognition Devices like Siri Alexa and Ok Google are not spying on us?

What are popular hobbies among Software Engineers?

Machine Learning Engineer Interview Questions and Answers

 

How many spaces is a tab in Java, Rust, C++, Python, C#, Powershell, Golang, Javascript

How many spaces is a tab in Java, Rust, C++, Python, C#, Powershell, Golang, Javascript

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

How many spaces is a tab in Java, Rust, C+, Python, C#, Powershell, Golang, etc.

A tab is not made out of spaces. It is a tab, whether in Java, Python, Rust, or generic text file editing. It is represented by a single Unicode character, U+0009.

It does not generally mean “insert this many spaces here” either. It means “put the cursor at the next closest tab stop in the line”. What that means exactly depends on the context. On an old typewriter I had, a tab key would advance the roller to the next column that was a multiple of 10

How many spaces is a tab in Java

That is pretty much the same function as the tab character does.

Just for reference, modern text editors have their tab stops set to either every 4 or every 8 characters. That doesn’t mean that 1 tab = 4/8 spaces, that means that putting in a tab will align the cursor with the next multiple of 4/8 columns

How many spaces is a tab in Java

In mainstream IDEs you can set the tab key to insert a desired number of spaces instead of a tab character.


 The concept of tab independent of space is rarely used these days. In any case, what the character represents is decoupled from what the key does is decoupled from what the screen shows.

In many IDEs, the tab character inserts the required number of spaces to advance to the next tab line. This is often a default.

I imagine it’s a compromise between tab loving extremists and space advocates. The ideal whitespace character is a subject of intense debate among programmers.


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

Source: Quora

  • (2023) Source Generators - C#
    by /u/fagnerbrack (programming) on March 18, 2024 at 6:13 pm

    submitted by /u/fagnerbrack [link] [comments]

  • 11 Ways to Optimize Kubernetes Registry Performance
    by /u/codingdecently (programming) on March 18, 2024 at 5:44 pm

    submitted by /u/codingdecently [link] [comments]

  • Creating a GUI application in Haskell
    by /u/emigs95 (programming) on March 18, 2024 at 5:43 pm

    submitted by /u/emigs95 [link] [comments]

  • Oh Sh*t, My App is Successful and I Didn’t Think About Accessibility
    by /u/jacobs-tech-tavern (programming) on March 18, 2024 at 5:21 pm

    submitted by /u/jacobs-tech-tavern [link] [comments]

  • Running PostgreSQL Inside Your Browser With PGLite
    by /u/Active-Fuel-49 (programming) on March 18, 2024 at 5:13 pm

    submitted by /u/Active-Fuel-49 [link] [comments]

  • C++ creator rebuts White House warning
    by /u/Franco1875 (programming) on March 18, 2024 at 5:07 pm

    submitted by /u/Franco1875 [link] [comments]

  • Empirically Supported Code Review Best Practices
    by /u/troikaman (programming) on March 18, 2024 at 4:26 pm

    submitted by /u/troikaman [link] [comments]

  • Introduction To Low Latency Programming: Clarify Program Scope
    by /u/theseawolfe (programming) on March 18, 2024 at 3:38 pm

    submitted by /u/theseawolfe [link] [comments]

  • xAI open sources their LLM - Grok
    by /u/bubblehack3r (programming) on March 18, 2024 at 3:35 pm

    submitted by /u/bubblehack3r [link] [comments]

  • Optimizing the Particle Life: From 400 to 4 million particles
    by /u/nucle4r_attack (programming) on March 18, 2024 at 3:33 pm

    submitted by /u/nucle4r_attack [link] [comments]

  • Implementing Vertical Form Controls
    by /u/feross (programming) on March 18, 2024 at 3:30 pm

    submitted by /u/feross [link] [comments]

  • Five Koans of Software Architecture
    by /u/tonystubblebine (programming) on March 18, 2024 at 3:08 pm

    submitted by /u/tonystubblebine [link] [comments]

  • We used Redis to solve our Dynamic Task Scheduling and Concurrent Execution issues, here's how
    by /u/Only_Piccolo5736 (programming) on March 18, 2024 at 2:12 pm

    submitted by /u/Only_Piccolo5736 [link] [comments]

  • mongo-playground - Visual Studio Marketplace
    by /u/Charming-Treacle-726 (programming) on March 18, 2024 at 1:49 pm

    submitted by /u/Charming-Treacle-726 [link] [comments]

  • Commentary: How to write a post-mortem that always blames Terry
    by /u/mixteenth (programming) on March 18, 2024 at 1:46 pm

    submitted by /u/mixteenth [link] [comments]

  • WebSockets vs Server-Sent-Events vs Long-Polling vs WebRTC vs WebTransport
    by /u/ApartmentWorking3164 (programming) on March 18, 2024 at 1:30 pm

    submitted by /u/ApartmentWorking3164 [link] [comments]

  • require(esm) in Node.js
    by /u/mariuz (programming) on March 18, 2024 at 1:16 pm

    submitted by /u/mariuz [link] [comments]

  • Zig, Rust, and other languages
    by /u/Xaneris47 (programming) on March 18, 2024 at 8:14 am

    submitted by /u/Xaneris47 [link] [comments]

  • JetBrains launches this new gen CI/CD with self-optimizing pipelines
    by /u/yegnau (programming) on March 18, 2024 at 8:12 am

    submitted by /u/yegnau [link] [comments]

  • Beginners Guide — Inner,Outer,left and right JOINS in SQL
    by Ashwini Shalke (Programming on Medium) on March 18, 2024 at 6:46 am

    What are Joins?Continue reading on Medium »

  • Top 5 ai programming languages 2024 You should know
    by pranjali AI (Programming on Medium) on March 18, 2024 at 6:25 am

    Python:Continue reading on Medium »

  • Yang mau jp siang ini Wa 081949831596 Link gacor : https://heylink.me/fika805
    by Fika Putri (Programming on Medium) on March 18, 2024 at 6:22 am

    Continue reading on Medium »

  • Complexity analysis of recursive functions.
    by John Anto (Programming on Medium) on March 18, 2024 at 6:19 am

    IntroductionContinue reading on Medium »

  • Simplifying Python Internationalization and Localization: A Practical Guide for Developers
    by Max N (Programming on Medium) on March 18, 2024 at 6:17 am

    Expand Your Python Applications’ Reach with Internationalization and Localization TechniquesContinue reading on Python in Plain English »

  • My Personal Tips for Not Getting Bored in Learning Programming
    by Josef Cruz (Programming on Medium) on March 18, 2024 at 6:17 am

    Even after doing it for a decade.Continue reading on JavaScript in Plain English »

  • 3 reasons why Devin will not replace programmers
    by AK (Programming on Medium) on March 18, 2024 at 6:11 am

    Lately, my timeline has been flooded with questions such as, “Will AI replace programmers in 2024?” or “Will AI take away my job?” These…Continue reading on Medium »

  • 7 Best CI/CD Tools for React in 2024
    by Jonathan Saring (Programming on Medium) on March 18, 2024 at 6:08 am

    Continuous Integration (CI) tools are pivotal in automating the software development process, especially for dynamic frameworks like React…Continue reading on Bits and Pieces »

  • Unveiling the Power of QuickBooks Cloud Hosting: Streamlining Financial Management in the Digital…
    by saksham singh (Programming on Medium) on March 18, 2024 at 5:55 am

    Small or huge agencies constantly seek ways to streamline operations and raise efficiency in a contemporary, fast-paced digital landscape…Continue reading on Medium »

  • Bloom Filters
    by /u/fagnerbrack (programming) on March 18, 2024 at 4:08 am

    submitted by /u/fagnerbrack [link] [comments]

  • Useful Uses of cat
    by /u/fagnerbrack (programming) on March 18, 2024 at 3:16 am

    submitted by /u/fagnerbrack [link] [comments]

  • announcing freenginx.org
    by /u/fagnerbrack (programming) on March 18, 2024 at 2:21 am

    submitted by /u/fagnerbrack [link] [comments]

  • JPEG DCT text lossifizer
    by /u/CrossFloss (programming) on March 17, 2024 at 8:32 pm

    submitted by /u/CrossFloss [link] [comments]

  • AWS Makes Cloud Formation Stack Creation up to 40% Faster
    by /u/rgancarz (programming) on March 17, 2024 at 6:09 pm

    submitted by /u/rgancarz [link] [comments]

  • The High-Risk Refactoring
    by /u/fagnerbrack (programming) on March 17, 2024 at 12:11 pm

    submitted by /u/fagnerbrack [link] [comments]

  • [META] The future of r/programming
    by /u/ketralnis (programming) on October 9, 2023 at 4:03 pm

    Hello fellow programs! tl;dr what should r/programming's rules be? And also a call for additional mods. We'll leave this stickied for a few days to gather feedback. Here are the broad categories of content that we see, along with whether they are currently allowed. ✅ means that it's currently allowed, 🚫 means that it's not currently allowed, ⚠️ means that we leave it up if it is already popular but if we catch it young in its life we do try to remove it early. ✅ Actual programming content. They probably have actual code in them. Language or library writeups, papers, technology descriptions. How an allocator works. How my new fancy allocator I just wrote works. How our startup built our Frobnicator, rocket ship emoji. For many years this was the only category of allowed content. ✅ Programming news. ChatGPT can write code. A big new CVE just dropped. Curl 8.01 released now with Coffee over IP support. ✅ Programmer career content. How to become a Staff engineer in 30 days. Habits of the best engineering managers. How to deal with your annoying coworkers, Jeff. ✅ Articles/news interesting to programmers but not about programming. Work from home is bullshit. Return to office is bullshit. There's a Steam sale on programming games. Terry Davis has died. How to SCRUMM. App Store commissions are going up. How to hire a more diverse development team. Interviewing programmers is broken. ⚠️ General technology news. Google buys its last competitor. A self driving car hit a pedestrian. Twitter is collapsing. Oculus accidentally showed your grandmother a penis. Github sued when Copilot produces the complete works of Harry Potter in a code comment. Meta cancels work from home. Gnome dropped a feature I like. How to run Stable Diffusion to generate pictures of, uh, cats, yeah it's definitely just for cats. A bitcoin VR metaversed my AI and now my app store is mobile social local. 🚫 Politics. The Pirate Party is winning in Sweden. Please vote for net neutrality. Big Tech is being sued in Europe for gestures broadly. 🚫 Gossip. Richard Stallman switches to Windows. Elon Musk farted. Linus Torvalds was a poopy-head on a mailing list. Grace Hopper Conference is now 60% male. The People's Rust Foundation is arguing with the Rust Foundation For The People. Terraform has been forked into Terra and Form. Stack Overflow sucks now. Stack Overflow is good actually. ✅ Demos with code. I wrote a game, here it is on GitHub 🚫 Demos without code. I wrote a game, come buy it! Please give me feedback on my startup (totally not an ad nosirree). I stayed up all night writing a commercial text editor, here's the pricing page. I made a DALL-E image generator. I made the fifteenth animation of A* this week, here's a GIF. 🚫 AskReddit type forum questions. What's your favourite programming language? Tabs or spaces? Does anyone else hate it when. 🚫 Support questions. How do I write a web crawler? How do I get into programming? Where's my missing semicolon? Please do this obvious homework problem for me. Personally I feel very strongly about not allowing these because they'd quickly drown out all of the actual content I come to see, and there are already much more effective places to get them answered anyway. In real life the quality of the ones that we see is also universally very low. 🚫 Surveys and 🚫 Job postings and anything else that is looking to extract value from a place a lot of programmers hang out without contributing anything itself. 🚫 Meta posts. DAE think r/programming sucks? Why did you remove my post? Why did you ban this user that is totes not me I swear I'm just asking questions. Except this meta post. This one is okay because I'm a tyrant that the rules don't apply to (I assume you are saying about me to yourself right now). 🚫 Images, memes, anything low-effort or low-content. Thankfully we very rarely see any of this so there's not much to remove but like support questions once you have a few of these they tend to totally take over because it's easier to make a meme than to write a paper and also easier to vote on a meme than to read a paper. ⚠️ Posts that we'd normally allow but that are obviously, unquestioningly super low quality like blogspam copy-pasted onto a site with a bazillion ads. It has to be pretty bad before we remove it and even then sometimes these are the first post to get traction about a news event so we leave them up if they're the best discussion going on about the news event. There's a lot of grey area here with CVE announcements in particular: there are a lot of spammy security "blogs" that syndicate stories like this. ⚠️ Posts that are duplicates of other posts or the same news event. We leave up either the first one or the healthiest discussion. ⚠️ Posts where the title editorialises too heavily or especially is a lie or conspiracy theory. Comments are only very loosely moderated and it's mostly 🚫 Bots of any kind (Beep boop you misspelled misspelled!) and 🚫 Incivility (You idiot, everybody knows that my favourite toy is better than your favourite toy.) However the number of obvious GPT comment bots is rising and will quickly become untenable for the number of active moderators we have. There are some topics such as Code of Conduct arguments within projects that I don't know where to place where we've been doing a civility check on the comments thread and using that to make the decision. Similarly some straddle the line (a link to a StackOverflow post asking for help and the reddit OP is the StackOverflow OP, but there's a lot of technical content and the reddit discussion is healthy). And even most 🚫s above are left up if there's a healthy discussion going already by the time we see it. So what now? We need to decide what r/programming should be about and we need to write those rules down so that mods can consistently apply them. The rules as written are pretty vague and the way we're moderating in practise is only loosely connected to them. We're looking for feedback on what kind of place r/programming should be so tell us below. We need additional mods. If you're interested in helping moderate please post below, saying why you'd be a good mod and what you'd would change about the space if you were. You don't need to be a moderator elsewhere but please do mention it if you are and what we could learn on r/programming that you already know. Currently I think I'm the only one going down the new page every morning and removing the rule-breaking posts. (Today these are mostly "how do I program computer" or "can somebody help me fix my printer", and obvious spam.) This results in a lot of threads complaining about the moderation quality and, well, it's not wrong. I'm not rigorously watching the mod queue and I'm not trawling comments threads looking for bad actors unless I'm in that thread anyway and I don't use reddit every single day. So if we want it to be better we'll need more human power. FAQ: Why do we need moderation at all? Can't the votes just do it? We know there is demand for unmoderated spaces in the world, but r/programming isn't that space. This is our theory on why keeping the subreddit on topic is important: Forums have the interesting property that whatever is on the front page today is what will be on the front page tomorrow. When a user comes to the site and sees a set of content, they believe that that's what this website is about. If they like it they'll stay and contribute that kind of content and if they don't like it they won't stay, leaving only the people that liked the content they saw yesterday. So the seed content is important and keeping things on topic is important. If you like r/programming then you need moderation to keep it the way that you like it (or make it be the place you wish it were) because otherwise entropic drift will make it be a different place. And once you have moderation it's going to have a subjective component, that's just the nature of it. Because of the way reddit works, on a light news day r/programming doesn't get enough daily content for articles to meaningfully compete with each other. Towards the end of the day if I post something to r/programming it will immediately go to the front page of all r/programming subscribers. So while it's true that sub-par and rule-breaking posts already do most of their damage before the mods even see them, the whole theory of posts competing via votes alone doesn't really work in a lower-volume subreddit. Because of the mechanics of moderation it's not really possible to allow the subreddit to be say 5% support questions. Even if we wanted to allow it to be a small amount of the conten, the individuals whose content was removed would experience and perceive this as a punitive action against them. That means that any category we allow could theoretically completely take over r/programming (like the career posts from last week) so we should only allow types of content that we'd be okay with taking it over. Personally my dream is for r/programming to be the place with the highest quality programming content, where I can go to read something interesting and learn something new every day. That dream is at odds with allowing every piece of blogspam and "10 ways to convince your boss to give you a raise, #2 will get you fired!" submitted by /u/ketralnis [link] [comments]

Machine Learning Engineer Interview Questions and Answers

Summary of Machine Learning and AI Capabilities

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Machine Learning Engineer Interview Questions and Answers

Learning: Supervised, Unsupervised, Reinforcement Learning

What is Machine Learning?

Machine learning is the study of computer algorithms that improve automatically through experience. It is seen as a subset of artificial intelligence. Machine Learning explores the study and construction of algorithms that can learn from and make predictions on data. You select a model to train and then manually perform feature extraction. Used to devise complex models and algorithms that lend themselves to a prediction which in commercial use is known as predictive analytics.

2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams
2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams

Below are the most common Machine Learning use cases and capabilities:

Summary of ML/AI Capabilities

Machine Learning For Dummies App


Machine Learning For Dummies

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

What is Supervised Learning? 

Supervised learning is the machine learning task of inferring a function from labeled training data. The training data consist of a set of training examples.

Algorithms: Support Vector Machines, Regression, Naive Bayes, Decision Trees, K-nearest Neighbor Algorithm and Neural Networks

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

Example: If you built a fruit classifier, the labels will be “this is an orange, this is an apple and this is a banana”, based on showing the classifier examples of apples, oranges and bananas.

What is Unsupervised learning?

Unsupervised learning is a type of machine learning algorithm used to draw inferences from datasets consisting of input data without labelled responses.

Algorithms: Clustering, Anomaly Detection, Neural Networks and Latent Variable Models

Example: In the same example, a fruit clustering will categorize as “fruits with soft skin and lots of dimples”, “fruits with shiny hard skin” and “elongated yellow fruits”.

Explain the difference between supervised and unsupervised machine learning?

In supervised machine learning algorithms, we have to provide labeled data, for example, prediction of stock market prices, whereas in unsupervised we need not have labeled data, for example, classification of emails into spam and non-spam.

What is deep learning, and how does it contrast with other machine learning algorithms?

Deep learning is a subset of machine learning that is concerned with neural networks: how to use backpropagation and certain principles from neuroscience to more accurately model large sets of unlabelled or semi-structured data. In that sense, deep learning represents an unsupervised learning algorithm that learns representations of data through the use of neural nets.

What is Problem Formulation in Machine Learning?

The problem formulation phase of the ML Pipeline is critical, and it’s where everything begins. Typically, this phase is kicked off with a question of some kind. Examples of these kinds of questions include: Could cars really drive themselves?  What additional product should we offer someone as they checkout? How much storage will clients need from a data center at a given time? 

The problem formulation phase starts by seeing a problem and thinking “what question, if I could answer it, would provide the most value to my business?” If I knew the next product a customer was going to buy, is that most valuable? If I knew what was going to be popular over the holidays, is that most valuable? If I better understood who my customers are, is that most valuable?

However, some problems are not so obvious. When sales drop, new competitors emerge, or there’s a big change to a company/team/org, it can be easy to say, “I see the problem!” But sometimes the problem isn’t so clear. Consider self-driving cars. How many people think to themselves, “driving cars is a huge problem”? Probably not many. In fact, there isn’t a problem in the traditional sense of the word but there is an opportunity. Creating self-driving cars is a huge opportunity. That doesn’t mean there isn’t a problem or challenge connected to that opportunity. How do you design a self-driving system? What data would you look at to inform the decisions you make? Will people purchase self-driving cars?

Part of the problem formulation phase includes seeing where there are opportunities to use machine learning.  

Djamgatech: Build the skills that’ll drive your career into six figures: Get Djamgatech.

To formulate a problem in ML, consider the following questions:

  1. Is machine learning appropriate for this problem, and why or why not?
  2. What is the ML problem if there is one, and what would a success metric look like?
  3. What kind of ML problem is this?
  4. Is the data appropriate?

Machine Learning Problem Formulation Examples:

1)  Amazon recently began advertising to its customers when they visit the company website. The Director in charge of the initiative wants the advertisements to be as tailored to the customer as possible. You will have access to all the data from the retail webpage, as well as all the customer data.

  • ML is appropriate because of the scale, variety and speed required. There are potentially thousands of ads and millions of customers that need to be served customized ads immediately as they arrive to the site.
  • The problem is ads that are not useful to customers are a wasted opportunity and a nuisance to customers, yet not serving ads at all is a wasted opportunity. So how does Amazon serve the most relevant advertisements to its retail customers?
    1. Success would be the purchase of a product that was advertised.
  • This is a supervised learning problem because we have a labeled data point, our success metric, which is the purchase of a product.
  • This data is appropriate because it is both the retail webpage data as well as the customer data.

What are the different Algorithm techniques in Machine Learning?

The different types of techniques in Machine Learning are
● Supervised Learning
● Unsupervised Learning
● Semi-supervised Learning
● Reinforcement Learning
● Transduction
● Learning to Learn

Machine Learning Engineer Interview Questions and Answers
Different Types of Machine Learning with examples

 

Machine Learning For Dummies App

Machine Learning For Dummies
Machine Learning For Dummies

 

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What’s the difference between a generative and discriminative model?

A generative model will learn categories of data while a discriminative model will simply learn the distinction between different categories of data. Discriminative models will generally outperform generative models on classification tasks.

What Are the Applications of Supervised Machine Learning in Modern Businesses?

Applications of supervised machine learning include:
Email Spam Detection
Here we train the model using historical data that consists of emails categorized as spam or not spam. This labeled information is fed as input to the model.
Healthcare Diagnosis
By providing images regarding a disease, a model can be trained to detect if a person is suffering from the disease or not.
Sentiment Analysis
This refers to the process of using algorithms to mine documents and determine whether they’re positive, neutral, or negative in sentiment.
Fraud Detection
Training the model to identify suspicious patterns, we can detect instances of possible fraud.

What Is Semi-supervised Machine Learning?

Ace the Microsoft Azure Fundamentals AZ-900 Certification Exam: Pass the Azure Fundamentals Exam with Ease

Supervised learning uses data that is completely labeled, whereas unsupervised learning uses no training data.
In the case of semi-supervised learning, the training data contains a small amount of labeled data and a large amount of unlabeled data.

What Are Unsupervised Machine Learning Techniques?

There are two techniques used in unsupervised learning: clustering and association.

Clustering
● Clustering problems involve data to be divided into subsets. These subsets, also called clusters, contain data that are similar to each other. Different clusters reveal different details about the objects, unlike classification or regression.

Association
● In an association problem, we identify patterns of associations between different variables or items.
● For example, an eCommerce website can suggest other items for you to buy, based on the prior purchases that you have made, spending habits, items in your wish list, other customers’ purchase habits, and so on.

What evaluation approaches would you work to gauge the effectiveness of a machine learning model?

You would first split the dataset into training and test sets, or perhaps use cross-validation techniques to further segment the dataset into composite sets of training and test sets within the data. You should then implement a choice selection of performance metrics: here is a fairly comprehensive list. You could use measures such as the F1 score, the accuracy, and the confusion matrix. What’s important here is to demonstrate that you understand the nuances of how a model is measured and how to choose the right performance measures for the right situations.

What Are the Three Stages of Building a Model in Machine Learning?

The three stages of building a machine learning model are:
● Model Building Choose a suitable algorithm for the model and train it according to the requirement
● Model Testing Check the accuracy of the model through the test data
● Applying the Mode Make the required changes after testing and use the final model for real-time projects. Here, it’s important to remember that once in a while, the model needs to be checked to make sure it’s working correctly. It should be modified to make sure that it is up-to-date.

A data scientist wants to visualize the correlation between features in their dataset. What tool(s) can they use to visualize this in a correlation matrix? 

Answer: Matplotlib, Seaborn

You are preprocessing a dataset that includes categorical features. You want to determine which categories of particular features are most common in your dataset. Which basic descriptive statistic could you use?
Answer: Mode

What are some examples of categorical features?

In machine learning and data science, categorical features are variables that can take on one of a limited number of values. For example, a categorical feature might represent the color of a car as Red, Yellow, or Blue. In general, categorical features are used to represent discrete characteristics (such as gender, race, or profession) that can be sorted into categories. When working with categorical features, it is often necessary to convert them into numerical form so that they can be used by machine learning algorithms. This process is known as encoding, and there are several different ways to encode categorical features. One common approach is to use a technique called one-hot encoding, which creates a new column for each possible category. For example, if there are three colors (Red, Yellow, and Blue), then each color would be represented by a separate column where all the values are either 0 or 1 (1 indicates that the row belongs to that category). Machine learning algorithms can then treat each column as a separate feature when training the model. Other approaches to encoding categorical data include label encoding and target encoding. These methods are often used in conjunction with one-hot encoding to improve the accuracy of machine learning models.

How many variables are enough for multiple regressions?

Which of the following is most suitable for supervised learning?

 
Answer: Identifying birds in an image
 
 
 

You’ve plotted the correlation matrix of your dataset’s features and realized that two of the features present a high negative correlation (-0.95). What should you do?

Answer: Remove one of the features

You are in charge of preprocessing the data your publishing company wants to use for a new ML model they’re building, which aims to predict the influence an academic journal will have in its field. The preprocessing step is necessary to prepare the data for model training. What type of issue with the data might you encounter during this preprocessing phase? 

Answer: Outliers, Missing values

A Machine Learning Engineer is creating and preparing data for a linear regression model. However, while preparing the data, the Engineer notices that about 20% of the numerical data contains missing values in the same two columns. The shape of the data is 500 rows by 4 columns, including the target column.
How can the Engineer handle the missing values in the data?

(Select TWO.)
 
 
 

Answer: Fill he missing values with mean of the column, Impute the missing values using regression

A Data Scientist created a correlation matrix between nine variables and the target variable. The correlation coefficient between two of the numerical variables, variable 1 and variable 5, is -0.95. How should the Data Scientist interpret the correlation coefficient?

Answer: As variable 1 increases, variable 5 decreases

An advertising and analytics company uses machine learning to predict user response to online advertisements using a custom XGBoost model. The company wants to improve its ML pipeline by porting its training and inference code, written in R, to Amazon SageMaker, and do so with minimal changes to the existing code.

Answer: Use the Build Your Own Container (BYOC) Amazon Sagemaker option.
Create a new docker container with the existing code. Register the container in Amazon Elastic Container registry. with the existing code. Register the container in Amazon Elastic Container Registry. Finally run the training and inference jobs using this container.

An ML engineer at a text analytics startup wants to develop a text classification model. The engineer collected large amounts of data to develop a supervised text classification model. The engineer is getting 99% accuracy on the dataset but when the model is deployed to production, it performs significantly worse. What is the most likely cause of this?

Answer: The engineer did not split the data to validate the model on unseen data.

For a classification problem, what does the loss function measure?
Answer: A loss function measures how accurate your prediction is with respect to the true values.

Gradient Descent is an important optimization method. What are 3 TRUE statements about the gradient descent method?

(Select THREE)
 
 

Answer: It tries to find the minimum of a loss function. It can involve multiple iterations
It uses learning rate to multiply the effect of gradients

What is Deep Learning?

Deep Learning is nothing but a paradigm of machine learning which has shown incredible promise in recent years. This is because of the fact that Deep Learning shows a great analogy with the functioning of the neurons in the human brain.

Machine Learning Engineer Interview Questions and Answers

Machine Learning For Dummies App

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is the difference between machine learning and deep learning?

Machine learning is a field of computer science that gives computers the ability to learn without being explicitly programmed. Machine learning can be categorized in the following four categories.
1. Supervised machine learning,
2. Semi-supervised machine learning,
3. Unsupervised machine learning,
4. Reinforcement learning.

Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks.

• The main difference between deep learning and machine learning is due to the way data is
presented in the system. Machine learning algorithms almost always require structured data, while deep learning networks rely on layers of ANN (artificial neural networks).

• Machine learning algorithms are designed to “learn” to act by understanding labeled data and then use it to produce new results with more datasets. However, when the result is incorrect, there is a need to “teach them”. Because machine learning algorithms require bulleted data, they are not suitable for solving complex queries that involve a huge amount of data.

• Deep learning networks do not require human intervention, as multilevel layers in neural
networks place data in a hierarchy of different concepts, which ultimately learn from their own mistakes. However, even they can be wrong if the data quality is not good enough.

• Data decides everything. It is the quality of the data that ultimately determines the quality of the result.

• Both of these subsets of AI are somehow connected to data, which makes it possible to represent a certain form of “intelligence.” However, you should be aware that deep learning requires much more data than a traditional machine learning algorithm. The reason for this is that deep learning networks can identify different elements in neural network layers only when more than a million data points interact. Machine learning algorithms, on the other hand, are capable of learning by pre-programmed criteria.

Can you explain the differences between supervised, unsupervised, and reinforcement learning?

In supervised learning, we train a model to learn the relationship between input data and output
data. We need to have labeled data to be able to do supervised learning.
With unsupervised learning, we only have unlabeled data. The model learns a representation of the data. Unsupervised learning is frequently used to initialize the parameters of the model when we have a lot of unlabeled data and a small fraction of labeled data. We first train an unsupervised model and, after that, we use the weights of the model to train a supervised model. In reinforcement learning, the model has some input data and a reward depending on the output of the model. The model learns a policy that maximizes the reward. Reinforcement learning has been applied successfully to strategic games such as Go and even classic Atari video games.

What is the reason for the popularity of Deep Learning in recent times? 

Now although Deep Learning has been around for many years, the major breakthroughs from these techniques came just in recent years. This is because of two main reasons:
• The increase in the amount of data generated through various sources
• The growth in hardware resources required to run these models
GPUs are multiple times faster and they help us build bigger and deeper deep learning models in comparatively less time than we required previously

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

Machine Learning For Dummies App

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

 

What is reinforcement learning?

 

Reinforcement Learning allows to take actions to max cumulative reward. It learns by trial and error through reward/penalty system. Environment rewards agent so by time agent makes better decisions.
Ex: robot=agent, maze=environment. Used for complex tasks (self-driving cars, game AI).

 

RL is a series of time steps in a Markov Decision Process:

 

1. Environment: space in which RL operates
2. State: data related to past action RL took
3. Action: action taken
4. Reward: number taken by agent after last action
5. Observation: data related to environment: can be visible or partially shadowed

 

Explain Ensemble learning.

In ensemble learning, many base models like classifiers and regressors are generated and combined together so that they give better results. It is used when we build component classifiers that are accurate and independent. There are sequential as well as parallel ensemble methods.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What are the parametric models? Give an example.

Parametric models are those with a finite number of parameters. To predict new data, you only need to know the parameters of the model. Examples include linear regression, logistic regression, and linear SVMs.
Non-parametric models are those with an unbounded number of parameters, allowing for more flexibility. To predict new data, you need to know the parameters of the model and the state of the data that has been observed. Examples include decision trees, k-nearest neighbors, and topic models using latent Dirichlet analysis.

What are support vector machines?

 

Support vector machines are supervised learning algorithms used for classification and regression analysis.

What is batch statistical learning?

Statistical learning techniques allow learning a function or predictor from a set of observed data that can make predictions about unseen or future data. These techniques provide guarantees on the performance of the learned predictor on the future unseen data based on a statistical assumption on the data generating process.

 

What Will Happen If the Learning Rate is Set inaccurately (Too Low or Too High)? 

 

When your learning rate is too low, training of the model will progress very slowly as we are making minimal updates to the weights. It will take many updates before reaching the minimum point.
If the learning rate is set too high, this causes undesirable divergent behavior to the loss function due to drastic updates in weights. It may fail to converge (model can give a good output) or even diverge (data is too chaotic for the network to train).

 

What Is The Difference Between Epoch, Batch, and Iteration in Deep Learning? 

 

Epoch – Represents one iteration over the entire dataset (everything put into the training model).
Batch – Refers to when we cannot pass the entire dataset into the neural network at once, so we divide the dataset into several batches.
Iteration – if we have 10,000 images as data and a batch size of 200. then an epoch should run 50 iterations (10,000 divided by 50).

 

Why Is Tensorflow the Most Preferred Library in Deep Learning?

Tensorflow provides both C++ and Python APIs, making it easier to work on and has a faster compilation time compared to other Deep Learning libraries like Keras and Torch. Tensorflow supports both CPU and GPU computing devices.

What Do You Mean by Tensor in Tensorflow?

A tensor is a mathematical object represented as arrays of higher dimensions. These arrays of data with different dimensions and ranks fed as input to the neural network are called “Tensors.”

Explain a Computational Graph.

Everything in TensorFlow is based on creating a computational graph. It has a network of nodes where each node operates, Nodes represent mathematical operations, and edges represent tensors. Since data flows in the form of a graph, it is also called a “DataFlow Graph.”

Cognition: Reasoning on top of data (Regression, Classification, Pattern Recognition)

What is the difference between classification and regression?

Classification is used to produce discrete results, classification is used to classify data into some specific categories. For example, classifying emails into spam and non-spam categories.
Whereas, We use regression analysis when we are dealing with continuous data, for example predicting stock prices at a certain point in time.

 

Explain the Bias-Variance Tradeoff.

Predictive models have a tradeoff between bias (how well the model fits the data) and variance (how much the model changes based on changes in the inputs).
Simpler models are stable (low variance) but they don’t get close to the truth (high bias).
More complex models are more prone to overfitting (high variance) but they are expressive enough to get close to the truth (low bias). The best model for a given problem usually lies somewhere in the middle.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is the difference between stochastic gradient descent (SGD) and gradient descent (GD)?

Both algorithms are methods for finding a set of parameters that minimize a loss function by evaluating parameters against data and then making adjustments.
In standard gradient descent, you’ll evaluate all training samples for each set of parameters.
This is akin to taking big, slow steps toward the solution.
In stochastic gradient descent, you’ll evaluate only 1 training sample for the set of parameters before updating them. This is akin to taking small, quick steps toward the solution.

How Can You Choose a Classifier Based on a Training Set Data Size?

When the training set is small, a model that has a right bias and low variance seems to work better because they are less likely to overfit. For example, Naive Bayes works best when the training set is large. Models with low bias and high variance tend to perform better as they work fine with complex relationships. 

 

Explain Latent Dirichlet Allocation (LDA)

Latent Dirichlet Allocation (LDA) is a common method of topic modeling, or classifying documents by subject matter.
LDA is a generative model that represents documents as a mixture of topics that each have their own probability distribution of possible words.
The “Dirichlet” distribution is simply a distribution of distributions. In LDA, documents are distributions of topics that are distributions of words.

Explain Principle Component Analysis (PCA)

PCA is a method for transforming features in a dataset by combining them into uncorrelated linear combinations.
These new features, or principal components, sequentially maximize the variance represented (i.e. the first principal component has the most variance, the second principal component has the second most, and so on).
As a result, PCA is useful for dimensionality reduction because you can set an arbitrary variance cutoff.

PCA is a dimensionality reduction technique that enables you to identify the correlations and patterns in the dataset so that it can be transformed into a dataset of significantly lower dimensions without any loss of important information.

• It is an unsupervised statistical technique used to examine the interrelations among a set of variables. It is also known as a general factor analysis where regression determines a line of best fit.

• It works on a condition that while the data in a higher-dimensional space is mapped to data in a lower dimension space, the variance or spread of the data in the lower dimensional space should be maximum.

PCA is carried out in the following steps

1. Standardization of Data
2. Computing the covariance matrix
3. Calculation of the eigenvectors and eigenvalues
4. Computing the Principal components
5. Reducing the dimensions of the Data.

a close up of text on a white background

Reference: Here

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What’s the F1 score? How would you use it?

The F1 score is a measure of a model’s performance. It is a weighted average of the precision and recall of a model, with results tending to 1 being the best, and those tending to 0 being the worst. You would use it in classification tests where true negatives don’t matter much.

When should you use classification over regression?

Classification produces discrete values and dataset to strict categories, while regression gives you continuous results that allow you to better distinguish differences between individual points.
You would use classification over regression if you wanted your results to reflect the belongingness of data points in your dataset to certain explicit categories (ex: If you wanted to know whether a name was male or female rather than just how correlated they were with male and female names.)

How do you ensure you’re not overfitting with a model?

This is a simple restatement of a fundamental problem in machine learning: the possibility of overfitting training data and carrying the noise of that data through to the test set, thereby providing inaccurate generalizations.
There are three main methods to avoid overfitting:
1- Keep the model simpler: reduce variance by taking into account fewer variables and parameters, thereby removing some of the noise in the training data.
2- Use cross-validation techniques such as k-folds cross-validation.
3- Use regularization techniques such as LASSO that penalize certain model parameters if they’re likely to cause overfitting.

How Will You Know Which Machine Learning Algorithm to Choose for Your Classification Problem?

While there is no fixed rule to choose an algorithm for a classification problem, you can follow these guidelines:
● If accuracy is a concern, test different algorithms and cross-validate them
● If the training dataset is small, use models that have low variance and high bias
● If the training dataset is large, use models that have high variance and little bias

Why is Area Under ROC Curve (AUROC) better than raw accuracy as an out-of-sample evaluation metric?

AUROC is robust to class imbalance, unlike raw accuracy.
For example, if you want to detect a type of cancer that’s prevalent in only 1% of the population, you can build a model that achieves 99% accuracy by simply classifying everyone has cancer-free.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What are the advantages and disadvantages of neural networks?

Advantages: Neural networks (specifically deep NNs) have led to performance breakthroughs for unstructured datasets such as images, audio, and video. Their incredible flexibility allows them to learn patterns that no other ML algorithm can learn.
Disadvantages: However, they require a large amount of training data to converge. It’s also difficult to pick the right architecture, and the internal “hidden” layers are incomprehensible.

Define Precision and Recall.

Precision
● Precision is the ratio of several events you can correctly recall to the total number of events you recall (mix of correct and wrong recalls).
● Precision = (True Positive) / (True Positive + False Positive)
Recall
● A recall is the ratio of a number of events you can recall the number of total events.
● Recall = (True Positive) / (True Positive + False Negative)

Model Evaluation with test data set

image

What Is Decision Tree Classification?

A decision tree builds classification (or regression) models as a tree structure, with datasets broken up into ever-smaller subsets while developing the decision tree, literally in a tree-like way with branches and nodes. Decision trees can handle both categorical and numerical data.

What Is Pruning in Decision Trees, and How Is It Done?

Pruning is a technique in machine learning that reduces the size of decision trees. It reduces the complexity of the final classifier, and hence improves predictive accuracy by the reduction of overfitting.
Pruning can occur in:
● Top-down fashion. It will traverse nodes and trim subtrees starting at the root
● Bottom-up fashion. It will begin at the leaf nodes
There is a popular pruning algorithm called reduced error pruning, in which:
● Starting at the leaves, each node is replaced with its most popular class
● If the prediction accuracy is not affected, the change is kept
● There is an advantage of simplicity and speed

What Is a Recommendation System?

Anyone who has used Spotify or shopped at Amazon will recognize a recommendation system:
It’s an information filtering system that predicts what a user might want to hear or see based on choice patterns provided by the user.

What Is Kernel SVM?

Kernel SVM is the abbreviated version of the kernel support vector machine. Kernel methods are a class of algorithms for pattern analysis, and the most common one is the kernel SVM.

What Are Some Methods of Reducing Dimensionality?

You can reduce dimensionality by combining features with feature engineering, removing collinear features, or using algorithmic dimensionality reduction.
Now that you have gone through these machine learning interview questions, you must have got an idea of your strengths and weaknesses in this domain.

 

How is KNN different from k-means clustering?

K-Nearest Neighbors is a supervised classification algorithm, while k-means clustering is an unsupervised clustering algorithm. While the mechanisms may seem similar at first, what this really means is that in order for K-Nearest Neighbors to work, you need labeled data you want to classify an unlabeled point into (thus the nearest neighbor part). K-means clustering requires only a set of unlabeled points and a threshold: the algorithm will take unlabeled points and gradually learn how to cluster them into groups by computing the mean of the distance between different points.

What are difference between Data Mining and Machine learning?

Machine learning relates to the study, design, and development of the algorithms that give computers the capability to learn without being explicitly programmed. While data mining can be defined as the process in which the unstructured data tries to extract knowledge or unknown interesting patterns. During this processing machine, learning algorithms are used.

Machine Learning For Dummies
Machine Learning For Dummies

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is “Naive” in a Naive Bayes?

Reference: Naive Bayes Classifier on Wikipedia

Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes’ theorem with the “naive” assumption of conditional independence between every pair of features given the value of the class variable. Bayes’ theorem states the following relationship, given class variable y and dependent feature vector X1through Xn:

Machine Learning Algorithms Naive Bayes

What is PCA (Principal Component Analysis)? When do you use it?

Reference: PCA on wikipedia

Principal component analysis (PCA) is a statistical method used in Machine Learning. It consists in projecting data in a higher dimensional space into a lower dimensional space by maximizing the variance of each dimension.

The process works as following. We define a matrix A with > rows (the single observations of a dataset – in a tabular format, each single row) and @ columns, our features. For this matrix we construct a variable space with as many dimensions as there are features. Each feature represents one coordinate axis. For each feature, the length has been standardized according to a scaling criterion, normally by scaling to unit variance. It is determinant to scale the features to a common scale, otherwise the features with a greater magnitude will weigh more in determining the principal components. Once plotted all the observations and computed the mean of each variable, that mean will be represented by a point in the center of our plot (the center of gravity). Then, we subtract each observation with the mean, shifting the coordinate system with the center in the origin. The best fitting line resulting is the line that best accounts for the shape of the point swarm. It represents the maximum variance direction in the data. Each observation may be projected onto this line in order to get a coordinate value along the PC-line. This value is known as a score. The next best-fitting line can be similarly chosen from directions perpendicular to the first.
Repeating this process yields an orthogonal basis in which different individual dimensions of the data are uncorrelated. These basis vectors are called principal components.

PCA is mostly used as a tool in exploratory data analysis and for making predictive models. It is often used to visualize genetic distance and relatedness between populations.

Machine Learning For Dummies

PCA is a technique that is used for reducing the dimensionality of a dataset while still preserving as much of the variance as possible. It is commonly used in machine learning and data science, as it can help to improve the performance of models by making the data easier to work with. In order to perform PCA on a dataset, there are a few pre-processing steps that need to be undertaken.

  • First, any features that are strongly correlated with each other should be removed, as PCA will not be effective in reducing the dimensionality of the data if there are strong correlations present.
  • Next, any features that contain missing values should be imputed, as PCA cannot be performed on data that contains missing values.
  • Finally, the data should be scaled so that all features are on the same scale; this is necessary because PCA is based on the variance of the data, and if the scales of the features are different then PCA will not be able to accurately identify which features are most important in terms of variance.
  • Once these pre-processing steps have been completed, PCA can be performed on the dataset.

Principal component analysis (PCA) is a statistical technique that is used to reduce the dimensionality of a dataset. PCA is often used as a pre-processing step in machine learning and data science, as it can help to improve the performance of models. In order to perform PCA on a dataset, the data must first be scaled and centered. Scaling ensures that all of the features are on the same scale, which is important for PCA. Centering means that the mean of each feature is zero. This is also important for PCA, as PCA is sensitive to changes in the mean of the data. Once the data has been scaled and centered, PCA can be performed by computing the eigenvectors and eigenvalues of the covariance matrix. These eigenvectors and eigenvalues can then be used to transform the data into a lower-dimensional space.

SVM (Support Vector Machine)  algorithm

Reference: SVM on wikipedia

Classifying data is a common task in machine learning. Suppose some given data points each belong to one of two classes, and the goal is to decide which class a new data point will be in. In the case of supportvector machines, a data point is viewed as a p-dimensional vector (a list of p numbers), and we want to know whether we can separate such points with a (p − 1)-dimensional hyperplane. This is called a linear classifier. There are many hyperplanes that might classify the data. One reasonable choice as the best hyperplane is the one that represents the largest separation, or margin, between the two classes. So, we
choose the hyperplane so that the distance from it to the nearest data point on each side is maximized. If such a hyperplane exists, it is known as the maximum-margin hyperplane and the linear classifier it defines is known as a maximum-margin classifier; or equivalently, the perceptron of optimal stability. The best hyper plane that divides the data is H3.

  • SVMs are helpful in text and hypertext categorization, as their application can significantly reduce the need for labeled training instances in both the standard inductive and transductive settings.
  • Some methods for shallow semantic parsing are based on support vector machines.
  • Classification of images can also be performed using SVMs. Experimental results show that SVMs achieve significantly higher search accuracy than traditional query refinement schemes after just three to four rounds of relevance feedback.
  • Classification of satellite data like SAR data using supervised SVM.
  • Hand-written characters can be recognized using SVM.

What are the support vectors in SVM? 

In the diagram, we see that the sketched lines mark the distance from the classifier (the hyper plane) to the closest data points called the support vectors (darkened data points). The distance between the two thin lines is called the margin.

To extend SVM to cases in which the data are not linearly separable, we introduce the hinge loss function, max (0, 1 – yi(w∙ xi − b)). This function is zero if x lies on the correct side of the margin. For data on the wrong side of the margin, the function’s value is proportional to the distance from the margin. 

What are the different kernels in SVM?

There are four types of kernels in SVM.
1. LinearKernel
2. Polynomial kernel
3. Radial basis kernel
4. Sigmoid kernel

Machine Learning For Dummies
Machine Learning For Dummies

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What are the most known ensemble algorithms? 

Reference: Ensemble Algorithms

The most popular trees are: AdaBoost, Random Forest, and  eXtreme Gradient Boosting (XGBoost).

AdaBoost is best used in a dataset with low noise, when computational complexity or timeliness of results is not a main concern and when there are not enough resources for broader hyperparameter tuning due to lack of time and knowledge of the user.

Random forests should not be used when dealing with time series data or any other data where look-ahead bias should be avoided, and the order and continuity of the samples need to be ensured. This algorithm can handle noise relatively well, but more knowledge from the user is required to adequately tune the algorithm compared to AdaBoost.

The main advantages of XGBoost is its lightning speed compared to other algorithms, such as AdaBoost, and its regularization parameter that successfully reduces variance. But even aside from the regularization parameter, this algorithm leverages a learning rate (shrinkage) and subsamples from the features like random forests, which increases its ability to generalize even further. However, XGBoost is more difficult to understand, visualize and to tune compared to AdaBoost and random forests. There is a multitude of hyperparameters that can be tuned to increase performance.

What are Artificial Neural Networks?

Artificial Neural networks are a specific set of algorithms that have revolutionized machine learning. They are inspired by biological neural networks. Neural Networks can adapt to changing the input, so the network generates the best possible result without needing to redesign the output criteria.

Artificial Neural Networks works on the same principle as a biological Neural Network. It consists of inputs which get processed with weighted sums and Bias, with the help of Activation Functions.

How Are Weights Initialized in a Network?

There are two methods here: we can either initialize the weights to zero or assign them randomly.

Initializing all weights to 0: This makes your model similar to a linear model. All the neurons and every layer perform the same operation, giving the same output and making the deep net useless.

Initializing all weights randomly: Here, the weights are assigned randomly by initializing them very close to 0. It gives better accuracy to the model since every neuron performs different computations. This is the most commonly used method.

What Is the Cost Function? 

Also referred to as “loss” or “error,” cost function is a measure to evaluate how good your model’s performance is. It’s used to compute the error of the output layer during backpropagation. We push that error backwards through the neural network and use that during the different training functions.
The most known one is the mean sum of squared errors.

Machine Learning For Dummies

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What Are Hyperparameters?

With neural networks, you’re usually working with hyperparameters once the data is formatted correctly.
A hyperparameter is a parameter whose value is set before the learning process begins. It determines how a network is trained and the structure of the network (such as the number of hidden units, the learning rate, epochs, batches, etc.).

What Are the Different Layers on CNN?

Reference: Layers of CNN 

The Convolutional neural networks are regularized versions of multilayer perceptron (MLP). They were developed based on the working of the neurons of the animal visual cortex.

The objective of using the CNN:

The idea is that you give the computer this array of numbers and it will output numbers that describe the probability of the image being a certain class (.80 for a cat, .15 for a dog, .05 for a bird, etc.). It works similar to how our brain works. When we look at a picture of a dog, we can classify it as such if the picture has identifiable features such as paws or 4 legs. In a similar way, the computer is able to perform image classification by looking for low-level features such as edges and curves and then building up to more abstract concepts through a series of convolutional layers. The computer uses low-level features obtained at the initial levels to generate high-level features such as paws or eyes to identify the object.

There are four layers in CNN:
1. Convolutional Layer – the layer that performs a convolutional operation, creating several smaller picture windows to go over the data.
2. Activation Layer (ReLU Layer) – it brings non-linearity to the network and converts all the negative pixels to zero. The output is a rectified feature map. It follows each convolutional layer.
3. Pooling Layer – pooling is a down-sampling operation that reduces the dimensionality of the feature map. Stride = how much you slide, and you get the max of the n x n matrix
4. Fully Connected Layer – this layer recognizes and classifies the objects in the image.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Machine Learning For Dummies

 

Machine Learning For Dummies

What Is Pooling on CNN, and How Does It Work?

Pooling is used to reduce the spatial dimensions of a CNN. It performs down-sampling operations to reduce the dimensionality and creates a pooled feature map by sliding a filter matrix over the input matrix.

What are Recurrent Neural Networks (RNNs)? 

Reference: RNNs

RNNs are a type of artificial neural networks designed to recognize the pattern from the sequence of data such as Time series, stock market and government agencies etc.

Recurrent Neural Networks (RNNs) add an interesting twist to basic neural networks. A vanilla neural network takes in a fixed size vector as input which limits its usage in situations that involve a ‘series’ type input with no predetermined size.

RNNs are designed to take a series of input with no predetermined limit on size. One could ask what’s\ the big deal, I can call a regular NN repeatedly too?

Sure can, but the ‘series’ part of the input means something. A single input item from the series is related to others and likely has an influence on its neighbors. Otherwise it’s just “many” inputs, not a “series” input (duh!).
Recurrent Neural Network remembers the past and its decisions are influenced by what it has learnt from the past. Note: Basic feed forward networks “remember” things too, but they remember things they learnt during training. For example, an image classifier learns what a “1” looks like during training and then uses that knowledge to classify things in production.
While RNNs learn similarly while training, in addition, they remember things learnt from prior input(s) while generating output(s). RNNs can take one or more input vectors and produce one or more output vectors and the output(s) are influenced not just by weights applied on inputs like a regular NN, but also by a “hidden” state vector representing the context based on prior input(s)/output(s). So, the same input could produce a different output depending on previous inputs in the series.

In summary, in a vanilla neural network, a fixed size input vector is transformed into a fixed size output vector. Such a network becomes “recurrent” when you repeatedly apply the transformations to a series of given input and produce a series of output vectors. There is no pre-set limitation to the size of the vector. And, in addition to generating the output which is a function of the input and hidden state, we update the hidden state itself based on the input and use it in processing the next input.

What is the role of the Activation Function?

The Activation function is used to introduce non-linearity into the neural network helping it to learn more complex function. Without which the neural network would be only able to learn linear function which is a linear combination of its input data. An activation function is a function in an artificial neuron that delivers an output based on inputs.

Machine Learning libraries for various purposes

What is an Auto-Encoder?

Reference: Auto-Encoder

Auto-encoders are simple learning networks that aim to transform inputs into outputs with the minimum possible error. This means that we want the output to be as close to input as possible. We add a couple of layers between the input and the output, and the sizes of these layers are smaller than the input layer. The auto-encoder receives unlabeled input which is then encoded to reconstruct the input. 

An autoencoder is a type of artificial neural network used to learn efficient data coding in an unsupervised manner. The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise”. Along with the reduction side, a reconstructing side is learnt, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input, hence its name. Several variants exist to the basic model, with the aim of forcing the learned representations of the input to assume useful properties.
Autoencoders are effectively used for solving many applied problems, from face recognition to acquiring the semantic meaning of words.

What is a Boltzmann Machine?

Boltzmann machines have a simple learning algorithm that allows them to discover interesting features that represent complex regularities in the training data. The Boltzmann machine is basically used to optimize the weights and the quantity for the given problem. The learning algorithm is very slow in networks with many layers of feature detectors. “Restricted Boltzmann Machines” algorithm has a single layer of feature detectors which makes it faster than the rest.

Machine Learning For Dummies

What Is Dropout and Batch Normalization?

Dropout is a technique of dropping out hidden and visible nodes of a network randomly to prevent overfitting of data (typically dropping 20 per cent of the nodes). It doubles the number of iterations needed to converge the network. It used to avoid overfitting, as it increases the capacity of generalization.

Batch normalization is the technique to improve the performance and stability of neural networks by normalizing the inputs in every layer so that they have mean output activation of zero and standard deviation of one

Why Is TensorFlow the Most Preferred Library in Deep Learning?

TensorFlow provides both C++ and Python APIs, making it easier to work on and has a faster compilation time compared to other Deep Learning libraries like Keras and PyTorch. TensorFlow supports both CPU and GPU computing devices.

What is Tensor in TensorFlow?

A tensor is a mathematical object represented as arrays of higher dimensions. Think of a n-D matrix. These arrays of data with different dimensions and ranks fed as input to the neural network are called “Tensors.”

What is the Computational Graph?

Everything in a TensorFlow is based on creating a computational graph. It has a network of nodes where each node operates. Nodes represent mathematical operations, and edges represent tensors. Since data flows in the form of a graph, it is also called a “DataFlow Graph.”

How is logistic regression done? 

Logistic regression measures the relationship between the dependent variable (our label of what we want to predict) and one or more independent variables (our features) by estimating probability using its underlying logistic function (sigmoid).

Explain the steps in making a decision tree. 

1. Take the entire data set as input
2. Calculate entropy of the target variable, as well as the predictor attributes
3. Calculate your information gain of all attributes (we gain information on sorting different objects from each other)
4. Choose the attribute with the highest information gain as the root node
5. Repeat the same procedure on every branch until the decision node of each branch is finalized
For example, let’s say you want to build a decision tree to decide whether you should accept or decline a job offer. The decision tree for this case is as shown:

It is clear from the decision tree that an offer is accepted if:
• Salary is greater than $50,000
• The commute is less than an hour
• Coffee is offered

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

How do you build a random forest model?

A random forest is built up of a number of decision trees. If you split the data into different packages and make a decision tree in each of the different groups of data, the random forest brings all those trees together.

Steps to build a random forest model:

1. Randomly select ; features from a total of = features where  k<< m
2. Among the ; features, calculate the node D using the best split point
3. Split the node into daughter nodes using the best split
4. Repeat steps two and three until leaf nodes are finalized
5. Build forest by repeating steps one to four for > times to create > number of trees

Differentiate between univariate, bivariate, and multivariate analysis. 

Univariate data contains only one variable. The purpose of the univariate analysis is to describe the data and find patterns that exist within it.

The patterns can be studied by drawing conclusions using mean, median, mode, dispersion or range, minimum, maximum, etc.

Bivariate data involves two different variables. The analysis of this type of data deals with causes and relationships and the analysis is done to determine the relationship between the two variables.

Here, the relationship is visible from the table that temperature and sales are directly proportional to each other. The hotter the temperature, the better the sales.

Multivariate data involves three or more variables, it is categorized under multivariate. It is similar to a bivariate but contains more than one dependent variable.

Example: data for house price prediction
The patterns can be studied by drawing conclusions using mean, median, and mode, dispersion or range, minimum, maximum, etc. You can start describing the data and using it to guess what the price of the house will be.

What are the feature selection methods used to select the right variables?

There are two main methods for feature selection.
Filter Methods
This involves:
• Linear discrimination analysis
• ANOVA
• Chi-Square
The best analogy for selecting features is “bad data in, bad answer out.” When we’re limiting or selecting the features, it’s all about cleaning up the data coming in.

Wrapper Methods
This involves:
• Forward Selection: We test one feature at a time and keep adding them until we get a good fit
• Backward Selection: We test all the features and start removing them to see what works
better
• Recursive Feature Elimination: Recursively looks through all the different features and how they pair together

Wrapper methods are very labor-intensive, and high-end computers are needed if a lot of data analysis is performed with the wrapper method.

You are given a data set consisting of variables with more than 30 percent missing values. How will you deal with them? 

If the data set is large, we can just simply remove the rows with missing data values. It is the quickest way; we use the rest of the data to predict the values.

For smaller data sets, we can impute missing values with the mean, median, or average of the rest of the data using pandas data frame in python. There are different ways to do so, such as: df.mean(), df.fillna(mean)

Other option of imputation is using KNN for numeric or classification values (as KNN just uses k closest values to impute the missing value).

Q76: How will you calculate the Euclidean distance in Python?

plot1 = [1,3]

plot2 = [2,5]

The Euclidean distance can be calculated as follows:

euclidean_distance = sqrt((plot1[0]-plot2[0])**2 + (plot1[1]- plot2[1])**2)

What are dimensionality reduction and its benefits? 

Dimensionality reduction refers to the process of converting a data set with vast dimensions into data with fewer dimensions (fields) to convey similar information concisely.

This reduction helps in compressing data and reducing storage space. It also reduces computation time as fewer dimensions lead to less computing. It removes redundant features; for example, there’s no point in storing a value in two different units (meters and inches).

How should you maintain a deployed model?

The steps to maintain a deployed model are (CREM):

1. Monitor: constant monitoring of all models is needed to determine their performance accuracy.
When you change something, you want to figure out how your changes are going to affect things.
This needs to be monitored to ensure it’s doing what it’s supposed to do.
2. Evaluate: evaluation metrics of the current model are calculated to determine if a new algorithm is needed.
3. Compare: the new models are compared to each other to determine which model performs the best.
4. Rebuild: the best performing model is re-built on the current state of data.

How can a time-series data be declared as stationery?

  1. The mean of the series should not be a function of time.

  1. The variance of the series should not be a function of time. This property is known as homoscedasticity.

  1. The covariance of the i th term and the (i+m) th term should not be a function of time.

‘People who bought this also bought…’ recommendations seen on Amazon are a result of which algorithm?

The recommendation engine is accomplished with collaborative filtering. Collaborative filtering explains the behavior of other users and their purchase history in terms of ratings, selection, etc.
The engine makes predictions on what might interest a person based on the preferences of other users. In this algorithm, item features are unknown.
For example, a sales page shows that a certain number of people buy a new phone and also buy tempered glass at the same time. Next time, when a person buys a phone, he or she may see a recommendation to buy tempered glass as well.

What is a Generative Adversarial Network?

Suppose there is a wine shop purchasing wine from dealers, which they resell later. But some dealers sell fake wine. In this case, the shop owner should be able to distinguish between fake and authentic wine. The forger will try different techniques to sell fake wine and make sure specific techniques go past the shop owner’s check. The shop owner would probably get some feedback from wine experts that some of the wine is not original. The owner would have to improve how he determines whether a wine is fake or authentic.
The forger’s goal is to create wines that are indistinguishable from the authentic ones while the shop owner intends to tell if the wine is real or not accurately.

• There is a noise vector coming into the forger who is generating fake wine.
• Here the forger acts as a Generator.
• The shop owner acts as a Discriminator.
• The Discriminator gets two inputs; one is the fake wine, while the other is the real authentic wine.
The shop owner has to figure out whether it is real or fake.

So, there are two primary components of Generative Adversarial Network (GAN) named:
1. Generator
2. Discriminator

The generator is a CNN that keeps keys producing images and is closer in appearance to the real images while the discriminator tries to determine the difference between real and fake images. The ultimate aim is to make the discriminator learn to identify real and fake images.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

You are given a dataset on cancer detection. You have built a classification model and achieved an accuracy of 96 percent. Why shouldn’t you be happy with your model performance? What can you do about it?

Cancer detection results in imbalanced data. In an imbalanced dataset, accuracy should not be based as a measure of performance. It is important to focus on the remaining four percent, which represents the patients who were wrongly diagnosed. Early diagnosis is crucial when it comes to cancer detection and can greatly improve a patient’s prognosis.

Hence, to evaluate model performance, we should use Sensitivity (True Positive Rate), Specificity (True Negative Rate), F measure to determine the class wise performance of the classifier.

We want to predict the probability of death from heart disease based on three risk factors: age, gender, and blood cholesterol level. What is the most appropriate algorithm for this case?

The most appropriate algorithm for this case is logistic regression.

After studying the behavior of a population, you have identified four specific individual types that are valuable to your study. You would like to find all users who are most similar to each individual type. Which algorithm is most appropriate for this study? 

As we are looking for grouping people together specifically by four different similarities, it indicates the value of k. Therefore, K-means clustering is the most appropriate algorithm for this study.

You have run the association rules algorithm on your dataset, and the two rules {banana, apple} => {grape} and {apple, orange} => {grape} have been found to be relevant. What else must be true? 

{grape, apple} must be a frequent itemset.

Your organization has a website where visitors randomly receive one of two coupons. It is also possible that visitors to the website will not receive a coupon. You have been asked to determine if offering a coupon to website visitors has any impact on their purchase decisions. Which analysis method should you use?

One-way ANOVA: in statistics, one-way analysis of variance is a technique that can be used to compare means of two or more samples. This technique can be used only for numerical response data, the “Y”, usually one variable, and numerical or categorical input data, the “X”, always one variable, hence “oneway”.
The ANOVA tests the null hypothesis, which states that samples in all groups are drawn from populations with the same mean values. To do this, two estimates are made of the population variance. The ANOVA produces an F-statistic, the ratio of the variance calculated among the means to the variance within the samples. If the group means are drawn from populations with the same mean values, the variance between the group means should be lower than the variance of the samples, following the central limit
theorem. A higher ratio therefore implies that the samples were drawn from populations with different mean values.

What are the feature vectors?

A feature vector is an n-dimensional vector of numerical features that represent an object. In machine learning, feature vectors are used to represent numeric or symbolic characteristics (called features) of an object in a mathematical way that’s easy to analyze.

What is root cause analysis?

Root cause analysis was initially developed to analyze industrial accidents but is now widely used in other areas. It is a problem-solving technique used for isolating the root causes of faults or problems. A factor is called a root cause if its deduction from the problem-fault-sequence averts the final undesirable event from recurring.

Do gradient descent methods always converge to similar points?

They do not, because in some cases, they reach a local minimum or a local optimum point. You would not reach the global optimum point. This is governed by the data and the starting conditions.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What are the different Deep Learning Frameworks?

PyTorch: PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook’s AI Research lab. It is free and open-source software released under the Modified BSD license.
TensorFlow: TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library and is also used for machine learning applications such as neural networks. Licensed by Apache License 2.0. Developed by Google Brain Team.
Microsoft Cognitive Toolkit: Microsoft Cognitive Toolkit describes neural networks as a series of computational steps via a directed graph.
Keras: Keras is an open-source neural-network library written in Python. It is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, R, Theano, or PlaidML. Designed to enable fast experimentation with deep neural networks, it focuses on being user-friendly, modular, and extensible. Licensed by MIT.

What are the different Deep Learning Frameworks?

PyTorch: PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook’s AI Research lab. It is free and open-source software released under the Modified BSD license.
TensorFlow: TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library and is also used for machine learning applications such as neural networks. Licensed by Apache License 2.0. Developed by Google Brain Team.
Microsoft Cognitive Toolkit: Microsoft Cognitive Toolkit describes neural networks as a series of computational steps via a directed graph.
Keras: Keras is an open-source neural-network library written in Python. It is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, R, Theano, or PlaidML. Designed to enable fast experimentation with deep neural networks, it focuses on being user-friendly, modular, and extensible. Licensed by MIT.

How Does an LSTM Network Work?

Reference: LTSM

Long-Short-Term Memory (LSTM) is a special kind of recurrent neural network capable of learning long-term dependencies, remembering information for long periods as its default behavior. There are three steps in an LSTM network:
• Step 1: The network decides what to forget and what to remember.
• Step 2: It selectively updates cell state values.
• Step 3: The network decides what part of the current state makes it to the output.

What Is a Multi-layer Perceptron (MLP)?

Reference: MLP

As in Neural Networks, MLPs have an input layer, a hidden layer, and an output layer. It has the same structure as a single layer perceptron with one or more hidden layers.

Perceptron is a single layer neural network and a multi-layer perceptron is called Neural Networks.
A (single layer) perceptron is a single layer neural network that works as a linear binary classifier. Being a single layer neural network, it can be trained without the use of more advanced algorithms like back propagation and instead can be trained by “stepping towards” your error in steps specified by a learning rate. When someone says perceptron, I usually think of the single layer version.

 

Machine Learning Multi-Layer Perceptron

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is exploding gradients? 

https://machinelearningmastery.com/exploding-gradients-in-neural-networks/

While training an RNN, if you see exponentially growing (very large) error gradients which accumulate and result in very large updates to neural network model weights during training, they’re known as exploding gradients. At an extreme, the values of weights can become so large as to overflow and result in NaN values. The explosion occurs through exponential growth by repeatedly multiplying gradients through the network layers that have values larger than 1.0.
This has the effect of your model is unstable and unable to learn from your training data.
There are some subtle signs that you may be suffering from exploding gradients during the training of your network, such as:
• The model is unable to get traction on your training data (e.g. poor loss).
• The model is unstable, resulting in large changes in loss from update to update.
• The model loss goes to NaN during training.
• The model weights quickly become very large during training.
• The error gradient values are consistently above 1.0 for each node and layer during training.

Solutions
1. Re-Design the Network Model:
a. In deep neural networks, exploding gradients may be addressed by redesigning the
network to have fewer layers. There may also be some benefit in using a smaller batch
size while training the network.
b. In RNNs, updating across fewer prior time steps during training, called truncated
Backpropagation through time, may reduce the exploding gradient problem.

2. Use Long Short-Term Memory Networks: In RNNs, exploding gradients can be reduced by using the Long Short-Term Memory (LSTM) memory units and perhaps related gated-type neuron structures. Adopting LSTM memory units is a new best practice for recurrent neural networks for sequence prediction.

3. Use Gradient Clipping: Exploding gradients can still occur in very deep Multilayer Perceptron networks with a large batch size and LSTMs with very long input sequence lengths. If exploding gradients are still occurring, you can check for and limit the size of gradients during the training of your network. This is called gradient clipping. Specifically, the values of the error gradient are checked against a threshold value and clipped or set to that threshold value if the error gradient exceeds the threshold.

4. Use Weight Regularization: another approach, if exploding gradients are still occurring, is to check the size of network weights and apply a penalty to the networks loss function for large weight values. This is called weight regularization and often an L1 (absolute weights) or an L2 (squared weights) penalty can be used.

Machine Learning For Dummies

What is vanishing gradients? 

While training an RNN, your slope can become either too small; this makes the training difficult. When the slope is too small, the problem is known as a Vanishing Gradient. It leads to long training times, poor performance, and low accuracy.
• Hyperbolic tangent and Sigmoid/Soft-max suffer vanishing gradient.
• RNNs suffer vanishing gradient, LSTM no (so it is perfect to predict stock prices). In fact, the propagation of error through previous layers makes the gradient get smaller so the weights are not updated.

Solutions
1. Choose RELU
2. Use LSTM (for RNNs)
3. Use ResNet (Residual Network) → after some layers, add x again: F(x) → ⋯ → F(x) + x
4. Multi-level hierarchy: pre-train one layer at the time through unsupervised learning, then fine-tune via backpropagation
5. Gradient checking: debugging strategy used to numerically track and assess gradients during training.

What is Gradient Descent?

Let’s first explain what a gradient is. A gradient is a mathematical function. When calculated on a point of a function, it gives the hyperplane (or slope) of the directions in which the function increases more. The gradient vector can be interpreted as the “direction and rate of fastest increase”. If the gradient of a function is non-zero at a point p, the direction of the gradient is the direction in which the function increases most quickly from p, and the magnitude of the gradient is the rate of increase in that direction.
Further, the gradient is the zero vector at a point if and only if it is a stationary point (where the derivative vanishes).
In Data Science, it simply measures the change in all weights with regard to the change in error, as we are partially derivating by w the loss function.

Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function.

 

Machine Learning Gradient Descent

The goal of the gradient descent is to minimize a given function which, in our case, is the loss function of the neural network. To achieve this goal, it performs two steps iteratively.
1. Compute the slope (gradient) that is the first-order derivative of the function at the current point
2. Move-in the opposite direction of the slope increase from the current point by the computed amount
So, the idea is to pass the training set through the hidden layers of the neural network and then update the parameters of the layers by computing the gradients using the training samples from the training dataset.
Think of it like this. Suppose a man is at top of the valley and he wants to get to the bottom of the valley.
So, he goes down the slope. He decides his next position based on his current position and stops when he gets to the bottom of the valley which was his goal.

• Gradient descent is an iterative optimization algorithm that is popular and it is a base for many other optimization techniques, which tries to obtain minimal loss in a model by tuning the weights/parameters in the objective function.

• Types of Gradient Descent:

  1. Batch Gradient Descent
  2. Stochastic Gradient Descent
  3. Mini Batch Gradient Descent

• Steps to achieve minimal loss:

  1. The first stage in gradient descent is to pick a starting value (a starting point) for w1, which is set to 0 by many algorithms.
  2. The gradient descent algorithm then calculates the gradient of the loss curve at the starting point.
  3. The gradient always points in the direction of steepest increase in the loss function. The gradient descent algorithm takes a step in the direction of the negative gradient in order to reduce loss as quickly as possible.
  4. To determine the next point along the loss function curve, the gradient descent algorithm adds some fraction of the gradient’s magnitude to the starting point and moves forward.
  5. The gradient descent then repeats this process, edging ever closer to the minimum.

What is vanishing gradients? 

While training an RNN, your slope can become either too small; this makes the training difficult. When the slope is too small, the problem is known as a Vanishing Gradient. It leads to long training times, poor performance, and low accuracy.
• Hyperbolic tangent and Sigmoid/Soft-max suffer vanishing gradient.
• RNNs suffer vanishing gradient, LSTM no (so it is perfect to predict stock prices). In fact, the propagation of error through previous layers makes the gradient get smaller so the weights are not updated.

Solutions
1. Choose RELU
2. Use LSTM (for RNNs)
3. Use ResNet (Residual Network) → after some layers, add x again: F(x) → ⋯ → F(x) + x
4. Multi-level hierarchy: pre-train one layer at the time through unsupervised learning, then fine-tune via backpropagation
5. Gradient checking: debugging strategy used to numerically track and assess gradients during training.

What is Back Propagation and Explain it Works. 

Back propagation is a training algorithm used for neural network. In this method, we update the weights of each layer from the last layer recursively, with the formula:

Machine Learning Back Propagation Formula

It has the following steps:
• Forward Propagation of Training Data (initializing weights with random or pre-assigned values)
• Gradients are computed using output weights and target
• Back Propagate for computing gradients of error from output activation
• Update the Weights

What are the variants of Back Propagation? 

Reference: Variants of back propagation

  • Stochastic Gradient Descent: In Batch Gradient Descent we were considering all the examples for every step of Gradient Descent. But what if our dataset is very huge. Deep learning models crave for data. The more the data the more chances of a model to be good. Suppose our dataset has 5 million examples, then just to take one step the model will have to calculate the gradients of all the 5 million examples. This does not seem an efficient way. To tackle this problem, we have Stochastic Gradient Descent. In Stochastic Gradient Descent (SGD), we consider just one example at a time to take a single step. We do the following steps in one epoch for SGD:
    1. Take an example
    2. Feed it to Neural Network
    3. Calculate its gradient
    4. Use the gradient we calculated in step 3 to update the weights
    5. Repeat steps 1–4 for all the examples in training dataset
    Since we are considering just one example at a time the cost will fluctuate over the training examples and it will not necessarily decrease. But in the long run, you will see the cost decreasing with fluctuations. Also, because the cost is so fluctuating, it will never reach the minimum, but it will keep dancing around it. SGD can be used for larger datasets. It converges faster when the dataset is large as it causes updates to the parameters more frequently.

 Stochastic Gradient Descent (SGD)

 

Stochastic Gradient Descent (SGD)

  • Batch Gradient Descent: all the training data is taken into consideration to take a single step. We take the average of the gradients of all the training examples and then use that mean gradient to update our parameters. So that’s just one step of gradient descent in one epoch. Batch Gradient Descent is great for convex or relatively smooth error manifolds. In this case, we move somewhat directly towards an optimum solution. The graph of cost vs epochs is also quite smooth because we are averaging over all the gradients of training data for a single step. The cost keeps on decreasing over the epochs.

 

Batch Gradient Descent

  • Mini-batch Gradient Descent: It’s one of the most popular optimization algorithms. It’s a variant of Stochastic Gradient Descent and here instead of single training example, mini batch of samples is used. Batch Gradient Descent can be used for smoother curves. SGD can be used when the dataset is large. Batch Gradient Descent converges directly to minima. SGD converges faster for larger datasets.
    But, since in SGD we use only one example at a time, we cannot implement the vectorized implementation on it. This can slow down the computations. To tackle this problem, a mixture of Batch Gradient Descent and SGD is used. Neither we use all the dataset all at once nor we use the single example at a time. We use a batch of a fixed number of training examples which is less than the actual dataset and call it a mini-batch. Doing this helps us achieve the advantages of both the former variants we saw. So, after creating the mini-batches of fixed size, we do the following steps in one epoch:
    1. Pick a mini-batch
    2. Feed it to Neural Network
    3. Calculate the mean gradient of the mini-batch
    4. Use the mean gradient we calculated in step 3 to update the weights
    5. Repeat steps 1–4 for the mini-batches we created
    Just like SGD, the average cost over the epochs in mini-batch gradient descent fluctuates because we are averaging a small number of examples at a time. So, when we are using the mini-batch gradient descent we are updating our parameters frequently as well as we can use vectorized implementation for faster computations.

While we continue to integrate ML systems in high-stakes environments such as medical settings, roads, command control centers, we need to ensure they do not cause the loss of life. How can you handle this?

By focusing on the following, which includes everything outside of just developing SOTA models, as well inclusion of key stakeholders.

🔹Robustness: Create models that are resilient to adversaries, unusual situations, and Black Swan events

🔹Monitoring: Detect malicious use, monitor predictions, and discover unexpected model functionality

🔹Alignment: Build models that represent and safely optimize hard-to-specify human values

🔹External Safety: Use ML to address risks to how ML systems are handled, such as cyber attacks

Machine Learning Unsolved Problems_ n_Safety

Download Unsolved Problems in ML Safety Here

You are given a data set. The data set has missing values that spread along 1 standard deviation from the median. What percentage of data would remain unaffected? Why?

Since the data is spread across the median, let’s assume it’s a normal distribution. We know, in a normal distribution, ~68% of the data lies in 1 standard deviation from mean (or mode, median), which leaves ~32% of the data unaffected. Therefore, ~32% of the data would remain unaffected by missing values.

What are PCA, KPCA, and ICA used for?

PCA (Principal Components Analysis), KPCA ( Kernel-based Principal Component Analysis) and ICA ( Independent Component Analysis) are important feature extraction techniques used for dimensionality reduction.

What is the bias-variance decomposition of classification error in the ensemble method?

The expected error of a learning algorithm can be decomposed into bias and variance. A bias term measures how closely the average classifier produced by the learning algorithm matches the target function. The variance term measures how much the learning algorithm’s prediction fluctuates for different training sets.

When is Ridge regression favorable over Lasso regression?

You can quote ISLR’s authors Hastie, Tibshirani who asserted that, in the presence of few variables with medium / large sized effect, use lasso regression. In presence of many variables with small/medium-sized effects, use ridge regression.
Conceptually, we can say, lasso regression (L1) does both variable selection and parameter shrinkage, whereas Ridge regression only does parameter shrinkage and end up including all the coefficients in the model. In the presence of correlated variables, ridge regression might be the preferred choice. Also, ridge regression works best in situations where the least square estimates have higher variance. Therefore, it depends on our model objective.

You’ve built a random forest model with 10000 trees. You got delighted after getting training error as 0.00. But, the validation error is 34.23. What is going on? Haven’t you trained your model perfectly?

The model has overfitted. Training error 0.00 means the classifier has mimicked the training data patterns to an extent, that they are not available in the unseen data. Hence, when this classifier was run on an unseen sample, it couldn’t find those patterns and returned predictions with higher error. In a random forest, it happens when we use a larger number of trees than necessary. Hence, to avoid this situation, we should tune the number of trees using cross-validation.

What is a convex hull?

In the case of linearly separable data, the convex hull represents the outer boundaries of the two groups of data points. Once the convex hull is created, we get maximum margin hyperplane (MMH) as a perpendicular bisector between two convex hulls. MMH is the line which attempts to create the greatest separation between two groups.

What do you understand by Type I vs Type II error?

Type I error is committed when the null hypothesis is true and we reject it, also known as a ‘False Positive’. Type II error is committed when the null hypothesis is false and we accept it, also known as ‘False Negative’.
In the context of the confusion matrix, we can say Type I error occurs when we classify a value as positive (1) when it is actually negative (0). Type II error occurs when we classify a value as negative (0) when it is actually positive(1).

In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbors. Why not manhattan distance?

We don’t use manhattan distance because it calculates distance horizontally or vertically only. It has dimension restrictions. On the other hand, the euclidean metric can be used in any space to calculate distance. Since the data points can be present in any dimension, euclidean distance is a more viable option.

Example: Think of a chessboard, the movement made by a bishop or a rook is calculated by manhattan distance because of their respective vertical & horizontal movements.

Do you suggest that treating a categorical variable as a continuous variable would result in a better predictive model?

For better predictions, the categorical variable can be considered as a continuous variable only when the variable is ordinal in nature.

OLS is to linear regression what the maximum likelihood is logistic regression. Explain the statement.

OLS and Maximum likelihood are the methods used by the respective regression methods to approximate the unknown parameter (coefficient) value. In simple words, Ordinary least square(OLS) is a method used in linear regression which approximates the parameters resulting in minimum distance between actual and predicted values. Maximum
Likelihood helps in choosing the values of parameters which maximizes the likelihood that the parameters are most likely to produce observed data.

When does regularization becomes necessary in Machine Learning?

Regularization becomes necessary when the model begins to overfit/underfit. This technique introduces a cost term for bringing in more features with the objective function. Hence, it tries to push the coefficients for many variables to zero and hence reduce the cost term. This helps to reduce model complexity so that the model can become better at predicting (generalizing).

What is Linear Regression?

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Linear Regression is a supervised Machine Learning algorithm. It is used to find the linear relationship between the dependent and the independent variables for predictive analysis.

• Linear regression assumes that the relationship between the features and the target vector is approximately linear. That is, the effect of the features on the target vector is constant.

• In linear regression, the target variable y is assumed to follow a linear function of one or more predictor variables plus some random error. The machine learning task is to estimate the parameters of this equation which can be achieved in two ways:

• The first approach is through the lens of minimizing loss. A common practice in machine learning is to choose a loss function that defines how well a model with a given set of parameters estimates the observed data. The most common loss function for linear regression is squared error loss.

• The second approach is through the lens of maximizing the likelihood. Another common practice in machine learning is to model the target as a random variable whose distribution depends on one or more parameters, and then find the parameters that maximize its likelihood.

No alternative text description for this image

Credit: Vikram K.

image

What is the Variance Inflation Factor?

Variance Inflation Factor (VIF) is the estimate of the volume of multicollinearity in a collection of many regression variables.
VIF = Variance of the model / Variance of the model with a single independent variable
We have to calculate this ratio for every independent variable. If VIF is high, then it shows the high collinearity of the independent variables.

We know that one hot encoding increases the dimensionality of a dataset, but label encoding doesn’t. How?

When we use one-hot encoding, there is an increase in the dimensionality of a dataset. The reason for the increase in dimensionality is that, for every class in the categorical variables, it forms a different variable.

What is a Decision Tree?

A decision tree is used to explain the sequence of actions that must be performed to get the desired output. It is a hierarchical diagram that shows the actions.

What is the Binarizing of data? How to Binarize?

In most of the Machine Learning Interviews, apart from theoretical questions, interviewers focus on the implementation part. So, this ML Interview Questions focused on the implementation of the theoretical concepts.
Converting data into binary values on the basis of threshold values is known as the binarizing of data. The values that are less than the threshold are set to 0 and the values that are greater than the threshold are set to 1.
This process is useful when we have to perform feature engineering, and we can also use it for adding unique features.

Machine Learning For Dummies
Machine Learning For Dummies

What is cross-validation?

Cross-validation is essentially a technique used to assess how well a model performs on a new independent dataset. The simplest example of cross-validation is when you split your data into two groups: training data and testing data, where you use the training data to build the model and the testing data to test the model.

• Cross-validation is a resampling procedure used to evaluate machine learning models on a limited data sample. The procedure has a single parameter called k that refers to the number of groups that a given data sample is to be split into. As such, the procedure is often called k-fold cross-validation.

• Cross-validation is primarily used in applied machine learning to estimate the skill of a machine learning model on unseen data. That is, to use a limited sample in order to estimate how the model is expected to perform in general when used to make predictions on data not used during the training of the model.

• It is a popular method because it is simple to understand and because it generally results in a less biased or less optimistic estimate of the model skill than other methods, such as a simple train/test split.

• Procedure for K-Fold Cross Validation:
1. Shuffle the dataset randomly.
2. Split the dataset into k groups

3. For each unique group:
a. Take the group as a holdout or test data set
b. Take the remaining groups as a training data set
c. Fit a model on the training set and evaluate it on the test set
d. Retain the evaluation score and discard the model

4. Summarize the skill of the model using the sample of model evaluation scores

No alternative text description for this image

Credit: Vikram K.

When would you use random forests Vs SVM and why?

There are a couple of reasons why a random forest is a better choice of the model than a support vector machine:
● Random forests allow you to determine the feature importance. SVM’s can’t do this.
● Random forests are much quicker and simpler to build than an SVM.
● For multi-class classification problems, SVMs require a one-vs-rest method, which is less scalable and more memory intensive.

What are the drawbacks of a linear model?

There are a couple of drawbacks of a linear model:
● A linear model holds some strong assumptions that may not be true in the application. It assumes a linear relationship, multivariate normality, no or little multicollinearity, no auto-correlation, and homoscedasticity
● A linear model can’t be used for discrete or binary outcomes.
● You can’t vary the model flexibility of a linear model.

 

While we continue to integrate ML systems in high-stakes environments such as medical settings, roads, command control centers, we need to ensure they do not cause the loss of life. How can you handle this?

By focusing on the following, which includes everything outside of just developing SOTA models, as well inclusion of key stakeholders.

🔹Robustness: Create models that are resilient to adversaries, unusual situations, and Black Swan events

🔹Monitoring: Detect malicious use, monitor predictions, and discover unexpected model functionality

🔹Alignment: Build models that represent and safely optimize hard-to-specify human values

🔹External Safety: Use ML to address risks to how ML systems are handled, such as cyber attacks

Machine Learning Unsolved Problems_ n_Safety

Download Unsolved Problems in ML Safety Here

You are given a data set. The data set has missing values that spread along 1 standard deviation from the median. What percentage of data would remain unaffected? Why?

Since the data is spread across the median, let’s assume it’s a normal distribution. We know, in a normal distribution, ~68% of the data lies in 1 standard deviation from mean (or mode, median), which leaves ~32% of the data unaffected. Therefore, ~32% of the data would remain unaffected by missing values.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Machine Learning For Dummies
Machine Learning For Dummies

What are PCA, KPCA, and ICA used for?

PCA (Principal Components Analysis), KPCA ( Kernel-based Principal Component Analysis) and ICA ( Independent Component Analysis) are important feature extraction techniques used for dimensionality reduction.

What are support vector machines?

 

Support vector machines are supervised learning algorithms used for classification and regression analysis.

What is batch statistical learning?

Statistical learning techniques allow learning a function or predictor from a set of observed data that can make predictions about unseen or future data. These techniques provide guarantees on the performance of the learned predictor on the future unseen data based on a statistical assumption on the data generating process.

What is the bias-variance decomposition of classification error in the ensemble method?

The expected error of a learning algorithm can be decomposed into bias and variance. A bias term measures how closely the average classifier produced by the learning algorithm matches the target function. The variance term measures how much the learning algorithm’s prediction fluctuates for different training sets.

When is Ridge regression favorable over Lasso regression?

You can quote ISLR’s authors Hastie, Tibshirani who asserted that, in the presence of few variables with medium / large sized effect, use lasso regression. In presence of many variables with small/medium-sized effects, use ridge regression.
Conceptually, we can say, lasso regression (L1) does both variable selection and parameter shrinkage, whereas Ridge regression only does parameter shrinkage and end up including all the coefficients in the model. In the presence of correlated variables, ridge regression might be the preferred choice. Also, ridge regression works best in situations where the least square estimates have higher variance. Therefore, it depends on our model objective.

You’ve built a random forest model with 10000 trees. You got delighted after getting training error as 0.00. But, the validation error is 34.23. What is going on? Haven’t you trained your model perfectly?

The model has overfitted. Training error 0.00 means the classifier has mimicked the training data patterns to an extent, that they are not available in the unseen data. Hence, when this classifier was run on an unseen sample, it couldn’t find those patterns and returned predictions with higher error. In a random forest, it happens when we use a larger number of trees than necessary. Hence, to avoid this situation, we should tune the number of trees using cross-validation.

What is a convex hull?

In the case of linearly separable data, the convex hull represents the outer boundaries of the two groups of data points. Once the convex hull is created, we get maximum margin hyperplane (MMH) as a perpendicular bisector between two convex hulls. MMH is the line which attempts to create the greatest separation between two groups.

What do you understand by Type I vs Type II error?

Type I error is committed when the null hypothesis is true and we reject it, also known as a ‘False Positive’. Type II error is committed when the null hypothesis is false and we accept it, also known as ‘False Negative’.
In the context of the confusion matrix, we can say Type I error occurs when we classify a value as positive (1) when it is actually negative (0). Type II error occurs when we classify a value as negative (0) when it is actually positive(1).

In k-means or kNN, we use euclidean distance to calculate the distance between nearest neighbors. Why not manhattan distance?

We don’t use manhattan distance because it calculates distance horizontally or vertically only. It has dimension restrictions. On the other hand, the euclidean metric can be used in any space to calculate distance. Since the data points can be present in any dimension, euclidean distance is a more viable option.

Example: Think of a chessboard, the movement made by a bishop or a rook is calculated by manhattan distance because of their respective vertical & horizontal movements.

Do you suggest that treating a categorical variable as a continuous variable would result in a better predictive model?

For better predictions, the categorical variable can be considered as a continuous variable only when the variable is ordinal in nature.

OLS is to linear regression wha the maximum likelihood is logistic regression. Explain the statement.

OLS and Maximum likelihood are the methods used by the respective regression methods to approximate the unknown parameter (coefficient) value. In simple words, Ordinary least square(OLS) is a method used in linear regression which approximates the parameters resulting in minimum distance between actual and predicted values. Maximum
Likelihood helps in choosing the values of parameters which maximizes the likelihood that the parameters are most likely to produce observed data.

When does regularization becomes necessary in Machine Learning?

Regularization becomes necessary when the model begins to overfit/underfit. This technique introduces a cost term for bringing in more features with the objective function. Hence, it tries to push the coefficients for many variables to zero and hence reduce the cost term. This helps to reduce model complexity so that the model can become better at predicting (generalizing).

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is Linear Regression?

Linear Regression is a supervised Machine Learning algorithm. It is used to find the linear relationship between the dependent and the independent variables for predictive analysis.

What is the Variance Inflation Factor?

Variance Inflation Factor (VIF) is the estimate of the volume of multicollinearity in a collection of many regression variables.
VIF = Variance of the model / Variance of the model with a single independent variable
We have to calculate this ratio for every independent variable. If VIF is high, then it shows the high collinearity of the independent variables.

We know that one hot encoding increases the dimensionality of a dataset, but label encoding doesn’t. How?

When we use one-hot encoding, there is an increase in the dimensionality of a dataset. The reason for the increase in dimensionality is that, for every class in the categorical variables, it forms a different variable.

What is a Decision Tree?

A decision tree is used to explain the sequence of actions that must be performed to get the desired output. It is a hierarchical diagram that shows the actions.

What is the Binarizing of data? How to Binarize?

In most of the Machine Learning Interviews, apart from theoretical questions, interviewers focus on the implementation part. So, this ML Interview Questions focused on the implementation of the theoretical concepts.
Converting data into binary values on the basis of threshold values is known as the binarizing of data. The values that are less than the threshold are set to 0 and the values that are greater than the threshold are set to 1.
This process is useful when we have to perform feature engineering, and we can also use it for adding unique features.

What is cross-validation?

Cross-validation is essentially a technique used to assess how well a model performs on a new independent dataset. The simplest example of cross-validation is when you split your data into two groups: training data and testing data, where you use the training data to build the model and the testing data to test the model.

When would you use random forests Vs SVM and why?

There are a couple of reasons why a random forest is a better choice of the model than a support vector machine:
● Random forests allow you to determine the feature importance. SVM’s can’t do this.
● Random forests are much quicker and simpler to build than an SVM.
● For multi-class classification problems, SVMs require a one-vs-rest method, which is less scalable and more memory intensive.

What are the drawbacks of a linear model?

There are a couple of drawbacks of a linear model:
● A linear model holds some strong assumptions that may not be true in the application. It assumes a linear relationship, multivariate normality, no or little multicollinearity, no auto-correlation, and homoscedasticity
● A linear model can’t be used for discrete or binary outcomes.
● You can’t vary the model flexibility of a linear model.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Do you think 50 small decision trees are better than a large one? Why?

Another way of asking this question is “Is a random forest a better model than a decision tree?”
And the answer is yes because a random forest is an ensemble method that takes many weak decision trees to make a strong learner. Random forests are more accurate, more robust, and less prone to overfitting. 

What is a kernel? Explain the kernel trick

A kernel is a way of computing the dot product of two vectors x and ᫣y in some (possibly very high dimensional) feature space, which is why kernel functions are sometimes called “generalized dot product”
The kernel trick is a method of using a linear classifier to solve a non-linear problem by transforming linearly inseparable data to linearly separable ones in a higher dimension.

State the differences between causality and correlation?

Causality applies to situations where one action, say X, causes an outcome, say Y, whereas Correlation is just relating one action (X) to another action(Y) but X does not necessarily cause Y.

What is the exploding gradient problem while using the backpropagation technique?

When large error gradients accumulate and result in large changes in the neural network weights during training, it is called the exploding gradient problem. The values of weights can become so large as to overflow and result in NaN values. This makes the model unstable and the learning of the model to stall just like the vanishing gradient problem.

What do you mean by Associative Rule Mining (ARM)?

Associative Rule Mining is one of the techniques to discover patterns in data like features (dimensions) which occur together and features (dimensions) which are correlated.

What is Marginalization? Explain the process.

Marginalization is summing the probability of a random variable X given the joint probability distribution of X with other variables. It is an application of the law of total probability.

Why is the rotation of components so important in Principle Component Analysis(PCA)?

Rotation in PCA is very important as it maximizes the separation within the variance obtained by all the components because of which interpretation of components would become easier. If the components are not rotated, then we need extended components to describe the variance of the components.

What is the difference between regularization and normalization?

Normalization adjusts the data; regularization adjusts the prediction function. If your data is on very different scales (especially low to high), you would want to normalize the data. Alter each column to have compatible basic statistics. This can be helpful to make sure there is no loss of accuracy. One of the goals of model training is to identify the signal and ignore the noise if the model is given free rein to minimize error, there is a possibility of suffering from overfitting.
Regularization imposes some control on this by providing simpler fitting functions over complex ones.

How does the SVM algorithm deal with self-learning?

SVM has a learning rate and expansion rate which takes care of this. The learning rate compensates or penalizes the hyperplanes for making all the wrong moves and expansion rate deals with finding the maximum separation area between classes.

How do you handle outliers in the data?

Outlier is an observation in the data set that is far away from other observations in the data set.
We can discover outliers using tools and functions like box plot, scatter plot, Z-Score, IQR score etc. and then handle them based on the visualization we have got. To handle outliers, we can cap at some threshold, use transformations to reduce skewness of the data and remove outliers if they are anomalies or errors.

What are some techniques used to find similarities in the recommendation system?

 

Pearson correlation and Cosine correlation are techniques used to find similarities in recommendation systems.

Why would you Prune your tree?

In the context of data science or AIML, pruning refers to the process of reducing redundant branches of a decision tree. Decision Trees are prone to overfitting, pruning the tree helps to reduce the size and minimizes the chances of overfitting. Pruning involves turning branches of a decision tree into leaf nodes and removing the leaf nodes from the original branch. It serves as a tool to perform the tradeoff.

What are some of the EDA Techniques?

Exploratory Data Analysis (EDA) helps analysts to understand the data better and forms the foundation of better models.
Visualization
● Univariate visualization
● Bivariate visualization
● Multivariate visualization
Missing Value Treatment – Replace missing values with Either Mean/Median Outlier Detection – Use Boxplot to identify the distribution of Outliers, then Apply IQR to set the boundary for IQR

What is data augmentation?

 

Data augmentation is a technique for synthesizing new data by modifying existing data in such a way that the target is not changed, or it is changed in a known way.
CV is one of the fields where data augmentation is very useful. There are many modifications that we can do to images:
● Resize
● Horizontal or vertical flip
● Rotate
● Add noise
● Deform
● Modify colors
Each problem needs a customized data augmentation pipeline. For example, on OCR, doing flips will change the text and won’t be beneficial; however, resizes and small rotations may help.

What is Inductive Logic Programming in Machine Learning (ILP)?

Inductive Logic Programming (ILP) is a subfield of machine learning which uses logic programming representing background knowledge and examples.

What is the difference between inductive machine learning and deductive machine learning?

The difference between inductive machine learning and deductive machine learning are as follows: machine-learning where the model learns by examples from a set of observed instances to draw a generalized conclusion whereas in deductive learning the model first draws the conclusion and then the conclusion is drawn.

What is the Difference between machine learning and deep learning?

 

Machine learning is a branch of computer science and a method to implement artificial intelligence. This technique provides the ability to automatically learn and improve from experiences without being explicitly programmed.
Deep learning can be said as a subset of machine learning. It is mainly based on the artificial neural network where data is taken as an input and the technique makes intuitive decisions using the artificial neural network.

What Are The Steps Involved In Machine Learning Project?

As you plan for doing a machine learning project. There are several important steps you must follow to achieve a good working model and they are data collection, data preparation, choosing a machine learning model, training the model, model evaluation, parameter tuning and lastly prediction.

What are Differences between Artificial Intelligence and Machine Learning?

Artificial intelligence is a broader prospect than machine learning. Artificial intelligence mimics the cognitive functions of the human brain. The purpose of AI is to carry out a task in an intelligent manner based on algorithms. On the other hand, machine learning is a subclass of artificial intelligence. To develop an autonomous machine in such a way so that it can learn without being explicitly programmed is the goal of machine learning.

What are the steps Needed to choose the Appropriate Machine Learning Algorithm for your Classification problem?

Firstly, you need to have a clear picture of your data, your constraints, and your problems before heading towards different machine learning algorithms. Secondly, you have to understand which type and kind of data you have because it plays a primary role in deciding which algorithm you have to use.

Following this step is the data categorization step, which is a two-step process – categorization by input and categorization by output. The next step is to understand your constraints; that is, what is your data storage capacity? How fast the prediction has to be? etc.

Finally, find the available machine learning algorithms and implement them wisely. Along with that, also try to optimize the hyperparameters which can be done in three ways – grid search, random search, and Bayesian optimization.

What is the Convex Function?

A convex function is a continuous function, and the value of the midpoint at every interval in its given domain is less than the numerical mean of the values at the two ends of the interval.

What’s the Relationship between True Positive Rate and Recall?

The True positive rate in machine learning is the percentage of the positives that have been properly acknowledged, and recall is just the count of the results that have been correctly identified and are relevant. Therefore, they are the same things, just having different names. It is also known as sensitivity.

What are some tools for parallelizing Machine Learning Algorithms?

Almost all machine learning algorithms are easy to serialize. Some of the basic tools for parallelizing are Matlab, Weka, R, Octave, or the Python-based sci-kit learn.

What is meant by Genetic Programming?

Genetic Programming (GP) is almost similar to an Evolutionary Algorithm, a subset of machine learning. Genetic programming software systems implement an algorithm that uses random mutation, a fitness function, crossover, and multiple generations of evolution to resolve a user-defined task. The genetic programming model is based on testing and choosing the best option among a set of results.

What is meant by Bayesian Networks?

Bayesian Networks also referred to as ‘belief networks’ or ‘casual networks’, are used to represent the graphical model for probability relationship among a set of variables.
For example, a Bayesian network can be used to represent the probabilistic relationships between diseases and symptoms. As per the symptoms, the network can also compute the probabilities of the presence of various diseases.
Efficient algorithms can perform inference or learning in Bayesian networks. Bayesian networks which relate the variables (e.g., speech signals or protein sequences) are called dynamic Bayesian networks.

 

Which are the two components of the Bayesian logic program?

A Bayesian logic program consists of two components:
● Logical It contains a set of Bayesian Clauses, which capture the qualitative structure of the domain.
● Quantitative It is used to encode quantitative information about the domain.

How is machine learning used in day-to-day life?

Most of the people are already using machine learning in their everyday life. Assume that you are engaging with the internet, you are actually expressing your preferences, likes, dislikes through your searches. All these things are picked up by cookies coming on your computer, from this, the behavior of a user is evaluated. It helps to increase the progress of a user through the internet and provide similar suggestions.
The navigation system can also be considered as one of the examples where we are using machine learning to calculate a distance between two places using optimization techniques.

What is Sampling. Why do we need it?

Sampling is a process of choosing a subset from a target population that would serve as its representative. We use the data from the sample to understand the pattern in the community as a whole. Sampling is necessary because often, we can not gather or process the complete data within a reasonable time.

What does the term decision boundary mean?

A decision boundary or a decision surface is a hypersurface which divides the underlying feature space into two subspaces, one for each class. If the decision boundary is a hyperplane, then the classes are linearly separable.

Define entropy?

Entropy is the measure of uncertainty associated with random variable Y. It is the expected number of bits required to communicate the value of the variable.

Indicate the top intents of machine learning?

The top intents of machine learning are stated below,
● The system gets information from the already established computations to give well-founded decisions and outputs.
● It locates certain patterns in the data and then makes certain predictions on it to provide answers on matters.

Highlight the differences between the Generative model and the Discriminative model?

The aim of the Generative model is to generate new samples from the same distribution and new data instances, Whereas, the Discriminative model highlights the differences between different kinds of data instances. It tries to learn directly from the data and then classifies the data.

Identify the most important aptitudes of a machine learning engineer?

Machine learning allows the computer to learn itself without being decidedly programmed. It helps the system to learn from experience and then improve from its mistakes. The intelligence system, which is based on machine learning, can learn from recorded data and past incidents.
In-depth knowledge of statistics, probability, data modelling, programming language, as well as CS, Application of ML Libraries and algorithms, and software design is required to become a successful machine learning engineer.

What is feature engineering? How do you apply it in the process of modelling?

Feature engineering is the process of transforming raw data into features that better represent the underlying problem to the predictive models, resulting in improved model accuracy on unseen data.

How can learning curves help create a better model?

Learning curves give the indication of the presence of overfitting or underfitting. In a learning curve, the training error and cross-validating error are plotted against the number of training data points.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Perception: Vision, Audio, Speech, Natural Language

NLP: TF-IDF helps you to establish what?

TFIDF helps to establish how important a particular word is in the context of the document corpus. TF-IDF takes into account the number of times the word appears in the document and offset by the number of documents that appear in the corpus.
– TF is the frequency of term divided by a total number of terms in the document.
– IDF is obtained by dividing the total number of documents by the number of documents containing the term and then taking the logarithm of that quotient.
– Tf.idf is then the multiplication of two values TF and IDF
 

List 10 use cases to be solved using NLP techniques?

● Sentiment Analysis
● Language Translation (English to German, Chinese to English, etc..)
● Document Summarization
● Question Answering
● Sentence Completion
● Attribute extraction (Key information extraction from the documents)
● Chatbot interactions
● Topic classification
● Intent extraction
● Grammar or Sentence correction
● Image captioning
● Document Ranking
● Natural Language inference

Which NLP model gives the best accuracy amongst the following: BERT, XLNET, GPT-2, ELMo

XLNET has given best accuracy amongst all the models. It has outperformed BERT on 20 tasks and achieves state of art results on 18 tasks including sentiment analysis, question answering, natural language inference, etc.

What is Naive Bayes algorithm, When we can use this algorithm in NLP?

Naive Bayes algorithm is a collection of classifiers which works on the principles of the Bayes’theorem. This series of NLP model forms a family of algorithms that can be used for a wide range of classification tasks including sentiment prediction, filtering of spam, classifying documents and more.
Naive Bayes algorithm converges faster and requires less training data. Compared to other discriminative models like logistic regression, Naive Bayes model  takes lesser time to train. This algorithm is perfect for use while working with multiple classes and text classification where the data is dynamic and changes frequently.

Explain Dependency Parsing in NLP?

Dependency Parsing, also known as Syntactic parsing in NLP is a process of assigning syntactic structure to a sentence and identifying its dependency parses. This process is crucial to understand the correlations between the “head” words in the syntactic structure.
The process of dependency parsing can be a little complex considering how any sentence can have more than one dependency parses. Multiple parse trees are known as ambiguities.
Dependency parsing needs to resolve these ambiguities in order to effectively assign a syntactic structure to a sentence.
Dependency parsing can be used in the semantic analysis of a sentence apart from the syntactic structuring.

What is text Summarization?

Text summarization is the process of shortening a long piece of text with its meaning and effect intact. Text summarization intends to create a summary of any given piece of text and outlines the main points of the document. This technique has improved in recent times and is capable of summarizing volumes of text successfully.
Text summarization has proved to a blessing since machines can summarize large volumes of text in no time which would otherwise be really time-consuming. There are two types of text summarization:
● Extraction-based summarization
● Abstraction-based summarization

What is NLTK? How is it different from Spacy?

NLTK or Natural Language Toolkit is a series of libraries and programs that are used for symbolic and statistical natural language processing. This toolkit contains some of the most powerful libraries that can work on different ML techniques to break down and understand human language. NLTK is used for Lemmatization, Punctuation, Character count, Tokenization, and Stemming.
The difference between NLTK and Spacey are as follows:
● While NLTK has a collection of programs to choose from, Spacey contains only the best suited algorithm for a problem in its toolkit
● NLTK supports a wider range of languages compared to Spacey (Spacey supports only 7 languages)
● While Spacey has an object-oriented library, NLTK has a string processing library
● Spacey can support word vectors while NLTK cannot

What is information extraction?

Information extraction in the context of Natural Language Processing refers to the technique of extracting structured information automatically from unstructured sources to ascribe meaning to it. This can include extracting information regarding attributes of entities, relationship between different entities and more. The various models of information extraction includes:
● Tagger Module
● Relation Extraction Module
● Fact Extraction Module
● Entity Extraction Module
● Sentiment Analysis Module
● Network Graph Module
● Document Classification & Language Modeling Module

What is Bag of Words?

Bag of Words is a commonly used model that depends on word frequencies or occurrences to train a classifier. This model creates an occurrence matrix for documents or sentences irrespective of its grammatical structure or word order.

What is Pragmatic Ambiguity in NLP?

Pragmatic ambiguity refers to those words which have more than one meaning and their use in any sentence can depend entirely on the context. Pragmatic ambiguity can result in multiple interpretations of the same sentence. More often than not, we come across sentences which have words with multiple meanings, making the sentence open to interpretation. This multiple interpretation causes ambiguity and is known as Pragmatic ambiguity in NLP.

What is a Masked Language Model?

Masked language models help learners to understand deep representations in downstream tasks by taking an output from the corrupt input. This model is often used to predict the words to be used in a sentence.

What are the best NLP Tools?

Some of the best NLP tools from open sources are:
● SpaCy
● TextBlob
● Textacy
● Natural language Toolkit
● Retext
● NLP.js
● Stanford NLP
● CogcompNLP

What is POS tagging?

Parts of speech tagging better known as POS tagging refers to the process of identifying specific words in a document and group them as part of speech, based on its context. POS tagging is also known as grammatical tagging since it involves understanding grammatical structures and identifying the respective component.
POS tagging is a complicated process since the same word can be different parts of speech depending on the context. The same generic process used for word mapping is quite ineffective for POS tagging because of the same reason.

What is NES?

Name entity recognition is more commonly known as NER is the process of identifying specific entities in a text document which are more informative and have a unique context. These often denote places, people, organizations, and more. Even though it seems like these entities are proper nouns, the NER process is far from identifying just the nouns. In fact, NER involves entity
chunking or extraction wherein entities are segmented to categorize them under different predefined classes. This step further helps in extracting information.

Explain the Masked Language Model?

Masked language modelling is the process in which the output is taken from the corrupted input.
This model helps the learners to master the deep representations in downstream tasks. You can predict a word from the other words of the sentence using this model.

What is pragmatic analysis in NLP?

Pragmatic Analysis: It deals with outside word knowledge, which means knowledge that is external to the documents and/or queries. Pragmatics analysis that focuses on what was described is reinterpreted by what it actually meant, deriving the various aspects of language that require real-world knowledge.

What is perplexity in NLP?

The word “perplexed” means “puzzled” or “confused”, thus Perplexity in general means the inability to tackle something complicated and a problem that is not specified. Therefore, Perplexity in NLP is a way to determine the extent of uncertainty in predicting some text.
In NLP, perplexity is a way of evaluating language models. Perplexity can be high and low; Low perplexity is ethical because the inability to deal with any complicated problem is less while high perplexity is terrible because the failure to deal with a complicated is high.

What is ngram in NLP?

N-gram in NLP is simply a sequence of n words, and we also conclude the sentences which appeared more frequently, for example, let us consider the progression of these three words:
● New York (2 gram)
● The Golden Compass (3 gram)
● She was there in the hotel (4 gram)
Now from the above sequence, we can easily conclude that sentence (a) appeared more frequently than the other two sentences, and the last sentence(c) is not seen that often. Now if we assign probability in the occurrence of an n-gram, then it will be advantageous. It would help in making next-word predictions and in spelling error corrections.

Explain differences between AI, Machine Learning and NLP

Why self-attention is awesome?

“In terms of computational complexity, self-attention layers are faster than recurrent layers when the sequence length n is smaller than the representation dimensionality d, which is most often the case with sentence representations used by state-of-the-art models in machine translations, such as word-piece and byte-pair representations.” — from Attention is all you need.

Machine Learning For Dummies  on iOs

 

Machine Learning For Dummies on Windows

 

Machine Learning For Dummies Web/Android 

 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What are stop words?

 

Stop words are said to be useless data for a search engine. Words such as articles, prepositions, etc. are considered as stop words. There are stop words such as was, were, is, am, the, a, an, how, why, and many more. In Natural Language Processing, we eliminate the stop words to understand and analyze the meaning of a sentence. The removal of stop words is one of the most important tasks for search engines. Engineers design the algorithms of search engines in such a way that they ignore the use of stop words. This helps show the relevant search result for a query.

What is Latent Semantic Indexing (LSI)?

Latent semantic indexing is a mathematical technique used to improve the accuracy of the information retrieval process. The design of LSI algorithms allows machines to detect the hidden (latent) correlation between semantics (words). To enhance information understanding, machines generate various concepts that associate with the words of a sentence.
The technique used for information understanding is called singular value decomposition. It is generally used to handle static and unstructured data. The matrix obtained for singular value decomposition contains rows for words and columns for documents. This method best suits to identify components and group them according to their types.
The main principle behind LSI is that words carry a similar meaning when used in a similar context.
Computational LSI models are slow in comparison to other models. However, they are good at contextual awareness that helps improve the analysis and understanding of a text or a document.

What are Regular Expressions?

A regular expression is used to match and tag words. It consists of a series of characters for matching strings.
Suppose, if A and B are regular expressions, then the following are true for them:
● If {ɛ} is a regular language, then ɛ is a regular expression for it.
● If A and B are regular expressions, then A + B is also a regular expression within the language {A, B}.
● If A and B are regular expressions, then the concatenation of A and B (A.B) is a regular expression.
● If A is a regular expression, then A* (A occurring multiple times) is also a regular expression.

What are unigrams, bigrams, trigrams, and n-grams in NLP?

When we parse a sentence one word at a time, then it is called a unigram. The sentence parsed two words at a time is a bigram.
When the sentence is parsed three words at a time, then it is a trigram. Similarly, n-gram refers to the parsing of n words at a time.

What are the steps involved in solving an NLP problem?

Below are the steps involved in solving an NLP problem:

1. Gather the text from the available dataset or by web scraping
2. Apply stemming and lemmatization for text cleaning
3. Apply feature engineering techniques
4. Embed using word2vec
5. Train the built model using neural networks or other Machine Learning techniques
6. Evaluate the model’s performance
7. Make appropriate changes in the model
8. Deploy the model

There have some various common elements of natural language processing. Those elements are very important for understanding NLP properly, can you please explain the same in details with an example?

There have a lot of components normally using by natural language processing (NLP). Some of the major components are explained below:
● Extraction of Entity: It actually identifying and extracting some critical data from the available information which help to segmentation of provided sentence on identifying each entity. It can help in identifying one human that it’s fictional or real, same kind of reality identification for any organization, events or any geographic location etc.
● The analysis in a syntactic way: it mainly helps for maintaining ordering properly of the available words.

In the case of processing natural language, we normally mentioned one common terminology NLP and binding every language with the same terminology properly. Please explain in details about this NLP terminology with an example?

This is the basic NLP Interview Questions asked in an interview. There have some several factors available in case of explaining natural language processing. Some of the key factors are given below:

● Vectors and Weights: Google Word vectors, length of TF-IDF, varieties documents, word vectors, TF-IDF.
● Structure of Text: Named Entities, tagging of part of speech, identifying the head of the sentence.
● Analysis of sentiment: Know about the features of sentiment, entities available for the sentiment, sentiment common dictionary.
● Classification of Text: Learning supervising, set off a train, set of validation in Dev, Set of define test, a feature of the individual text, LDA.
● Reading of Machine Language: Extraction of the possible entity, linking with an individual entity, DBpedia, some libraries like Pikes or FRED.

Explain briefly about word2vec

Word2Vec embeds words in a lower-dimensional vector space using a shallow neural network.
The result is a set of word-vectors where vectors close together in vector space have similar meanings based on context, and word-vectors distant to each other have differing meanings. For example, apple and orange would be close together and apple and gravity would be relatively far.
There are two versions of this model based on skip-grams (SG) and continuous-bag-of-words (CBOW).

What are the metrics used to test an NLP model?

Accuracy, Precision, Recall and F1. Accuracy is the usual ratio of the prediction to the desired output. But going just be accuracy is naive considering the complexities involved.

What are some ways we can preprocess text input?

Here are several preprocessing steps that are commonly used for NLP tasks:
● case normalization: we can convert all input to the same case (lowercase or uppercase) as a way of reducing our text to a more canonical form
● punctuation/stop word/white space/special characters removal: if we don’t think these words or characters are relevant, we can remove them to reduce the feature space
● lemmatizing/stemming: we can also reduce words to their inflectional forms (i.e. walks → walk) to further trim our vocabulary
● generalizing irrelevant information: we can replace all numbers with a <NUMBER> token or all names with a <NAME> token.

How does the encoder-decoder structure work for language modelling?

The encoder-decoder structure is a deep learning model architecture responsible for several state of the art solutions, including Machine Translation.
The input sequence is passed to the encoder where it is transformed to a fixed-dimensional vector representation using a neural network. The transformed input is then decoded using another neural network. Then, these outputs undergo another transformation and a SoftMax layer. The final output is a vector of probabilities over the vocabularies. Meaningful information is extracted based on these probabilities.

How would you implement an NLP system as a service, and what are some pitfalls you might face in production?

This is less of a NLP question than a question for productionizing machine learning models. There are however certain intricacies to NLP models.

Without diving too much into the productionization aspect, an ideal Machine Learning service will have:
● endpoint(s) that other business systems can use to make inference
● a feedback mechanism for validating model predictions
● a database to store predictions and ground truths from the feedback
● a workflow orchestrator which will (upon some signal) re-train and load the new model for
serving based on the records from the database + any prior training data
● some form of model version control to facilitate rollbacks in case of bad deployments
● post-production accuracy and error monitoring

What are attention mechanisms and why do we use them?

This was a follow-up to the encoder-decoder question. Only the output from the last time step is passed to the decoder, resulting in a loss of information learned at previous time steps. This information loss is compounded for longer text sequences with more time steps.
Attention mechanisms are a function of the hidden weights at each time step. When we use attention in encoder-decoder networks, the fixed-dimensional vector passed to the decoder becomes a function of all vectors outputted in the intermediary steps.
Two commonly used attention mechanisms are additive attention and multiplicative attention. As the names suggest, additive attention is a weighted sum while multiplicative attention is a weighted multiplier of the hidden weights. During the training process, the model also learns weights for the attention mechanisms to recognize the relative importance of each time step.

How can we handle misspellings for text input?

By using word embeddings trained over a large corpus (for instance, an extensive web scrape of billions of words), the model vocabulary would include common misspellings by design. The model can then learn the relationship between misspelled and correctly spelled words to recognize their semantic similarity.
We can also preprocess the input to prevent misspellings. Terms not found in the model vocabulary can be mapped to the “closest” vocabulary term using:
● edit distance between strings
● phonetic distance between word pronunciations
● keyword distance to catch common typos

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is the problem with ReLu?

● Exploding gradient(Solved by gradient clipping)
● Dying ReLu — No learning if the activation is 0 (Solved by parametric relu)
● Mean and variance of activations is not 0 and 1.(Partially solved by subtracting around 0.5 from activation. Better explained in fastai videos)

What is the difference between learning latent features using SVD and getting embedding vectors using deep network?

SVD uses linear combination of inputs while a neural network uses nonlinear combination.

What is the information in the hidden and cell state of LSTM?

Hidden stores all the information till that time step and cell state stores particular information that might be needed in the future time step.

When is self-attention not faster than recurrent layers?

When the sequence length is greater than the representation dimensions. This is rare.

What is the benefit of learning rate warm-up?

Learning rate warm-up is a learning rate schedule where you have low (or lower) learning rate at the beginning of training to avoid divergence due to unreliable gradients at the beginning. As the model becomes more stable, the learning rate would increase to speed up convergence.

What’s the difference between hard and soft parameter sharing in multi-task learning?

What’s the difference between BatchNorm and LayerNorm?

BatchNorm computes the mean and variance at each layer for every minibatch whereas LayerNorm computes the mean and variance for every sample for each layer independently.

Hard sharing is where we train for all the task at the same time and update our weights using all the losses whereas soft sharing is where we train for one task at a time.

Batch normalisation allows you to set higher learning rates, increasing speed of training as it reduces the unstability of initial starting weights.

Difference between BatchNorm and LayerNorm?

BatchNorm — Compute the mean and var at each layer for every minibatch
LayerNorm — Compute the mean and var for every single sample for each layer independently

Why does the transformer block have LayerNorm instead of BatchNorm?

Looking at the advantages of LayerNorm, it is robust to batch size and works better as it works at the sample level and not batch level.

What changes would you make to your deep learning code if you knew there are errors in your training data?

We can do label smoothening where the smoothening value is based on % error. If any particular class has known error, we can also use class weights to modify the loss.

What are the tricks used in ULMFiT? (Not a great questions but checks the awareness)
● LM tuning with task text
● Weight dropout
● Discriminative learning rates for layers
● Gradual unfreezing of layers
● Slanted triangular learning rate schedule
This can be followed up with a question on explaining how they help.

Tell me a language model which doesn’t use dropout

ALBERT v2 — This throws a light on the fact that a lot of assumptions we take for granted are not necessarily true. The regularization effect of parameter sharing in ALBERT is so strong that dropouts are not needed. (ALBERT v1 had dropouts.)

What are the differences between GPT and GPT-2?

● Layer normalization was moved to the input of each sub-block, similar to a residual unit of type “building block” (differently from the original type “bottleneck”, it has batch normalization applied before weight layers).
● An additional layer normalization was added after the final self-attention block.
● A modified initialization was constructed as a function of the model depth.
● The weights of residual layers were initially scaled by a factor of 1/√n where n is the number of residual layers.
● Use larger vocabulary size and context size.

What are the differences between GPT and BERT?

● GPT is not bidirectional and has no concept of masking
● BERT adds next sentence prediction task in training and so it also has a segment embedding

What are the differences between BERT and ALBERT v2?

● Embedding matrix factorisation(helps in reducing no. of parameters)
● No dropout
● Parameter sharing(helps in reducing no. of parameters and regularisation)

How does parameter sharing in ALBERT affect the training and inference time?

No effect. Parameter sharing just decreases the number of parameters.

How would you reduce the inference time of a trained NN model?

● Serve on GPU/TPU/FPGA
● 16 bit quantisation and served on GPU with fp16 support
● Pruning to reduce parameters
● Knowledge distillation (To a smaller transformer model or simple neural network)
● Hierarchical softmax/Adaptive softmax
● You can also cache results as explained here.

Would you use BPE with classical models?

Of course! BPE is a smart tokeniser and it can help us get a smaller vocabulary which can help us find a model with less parameters.

How would you make an arxiv papers search engine? 

How would you make a plagiarism detector?

Get top k results with TF-IDF similarity and then rank results with
● semantic encoding + cosine similarity
● a model trained for ranking

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

How would you make a sentiment classifier?

This is a trick question. The interviewee can say all things such as using transfer learning and latest models but they need to talk about having a neutral class too otherwise you can have really good accuracy/f1 and still, the model will classify everything into positive or negative.
The truth is that a lot of news is neutral and so the training needs to have this class. The interviewee should also talk about how he will create a dataset and his training strategies like the selection of language model, language model fine-tuning and using various datasets for multitask learning.

What is the difference between regular expression and regular grammar?

A regular expression is the representation of natural language in the form of mathematical expressions containing a character sequence. On the other hand, regular grammar is the generator of natural language, defining a set of defined rules and syntax which the strings in the natural language must follow.

Why should we use Batch Normalization?

Once the interviewer has asked you about the fundamentals of deep learning architectures, they would move on to the key topic of improving your deep learning model’s performance.
Batch Normalization is one of the techniques used for reducing the training time of our deep learning algorithm. Just like normalizing our input helps improve our logistic regression model, we can normalize the activations of the hidden layers in our deep learning model as well:

How is backpropagation different in RNN compared to ANN?

In Recurrent Neural Networks, we have an additional loop at each node:
This loop essentially includes a time component into the network as well. This helps in capturing sequential information from the data, which could not be possible in a generic artificial neural network.
This is why the backpropagation in RNN is called Backpropagation through Time, as in backpropagation at each time step.

Which of the following is a challenge when dealing with computer vision problems?

Variations due to geometric changes (like pose, scale, etc), Variations due to photometric factors (like illumination, appearance, etc) and Image occlusion. All the above-mentioned options are challenges in computer vision.

Consider an image with width and height as 100×100. Each pixel in the image can have a color from Grayscale, i.e. values. How much space would this image require for storing?

The answer will be 8x100x100 because 8 bits will be required to represent a number from 0-256

Why do we use convolutions for images rather than just FC layers?

Firstly, convolutions preserve, encode, and actually use the spatial information from the image. If we used only FC layers we would have no relative spatial information. Secondly, Convolutional Neural Networks (CNNs) have a partially built-in translation in-variance, since each convolution kernel acts as it’s own filter/feature detector

What makes CNN’s translation-invariant?

As explained above, each convolution kernel acts as it’s own filter/feature detector. So let’s say you’re doing object detection, it doesn’t matter where in the image the object is since we’re going to apply the convolution in a sliding window fashion across the entire image anyways.

Why do we have max-pooling in classification CNNs?

Max-pooling in a CNN allows you to reduce computation since your feature maps are smaller after the pooling. You don’t lose too much semantic information since you’re taking the maximum activation. There’s also a theory that max-pooling contributes a bit to giving CNN’s more translation in-variance. Check out this great video from Andrew Ng on the benefits of max-pooling.

Why do segmentation CNN’s typically have an encoder-decoder style/structure?

The encoder CNN can basically be thought of as a feature extraction network, while the decoder uses that information to predict the image segments by “decoding” the features and upscaling to the original image size.

What is the significance of Residual Networks?

The main thing that residual connections did was allow for direct feature access from previous layers. This makes information propagation throughout the network much easier. One very interesting paper about this shows how using local skip connections gives the network a type of ensemble multi-path structure, giving features multiple paths to propagate throughout the network.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is batch normalization and why does it work?

Training Deep Neural Networks is complicated by the fact that the distribution of each layer’s inputs changes during training, as the parameters of the previous layers change. The idea is then to normalize the inputs of each layer in such a way that they have a mean output activation of zero and a standard deviation of one. This is done for each individual mini-batch at each layer i.e compute the mean and variance of that mini-batch alone, then normalize. This is analogous to how the inputs to networks are standardized. How does this help? We know that normalizing the inputs to a network helps it learn.
But a network is just a series of layers, where the output of one layer becomes the input to the next. That means we can think of any layer in a neural network as the first layer of a smaller subsequent network. Thought of as a series of neural networks feeding into each other, we normalize the output of one layer before applying the activation function and then feed it into the following layer (sub-network).

Why would you use many small convolutional kernels such as 3×3 rather than a few large ones?

This is very well explained in the VGGNet paper.

There are 2 reasons: First, you can use several smaller kernels rather than few large ones to get the same receptive field and capture more spatial context, but with the smaller kernels you are using less parameters and computations. Secondly, because with smaller kernels you will be using more filters, you’ll be able to use more activation functions and thus have a more discriminative mapping function being learned by your CNN.

What is Precision?

Precision (also called positive predictive value) is the fraction of relevant instances among the retrieved instances
Precision = true positive / (true positive + false positive)

What is Recall?

Recall (also known as sensitivity) is the fraction of relevant instances that have been retrieved over the total amount of relevant instances.
Recall = true positive / (true positive + false negative)

Define F1-score.

It is the weighted average of precision and recall. It considers both false positive and false negatives into account. It is used to measure the model’s performance.

What is cost function?

The cost function is a scalar function that Quantifies the error factor of the Neural Network. Lower the cost function better than the Neural network. Eg: MNIST Data set to classify the image, the input image is digit 2 and the Neural network wrongly predicts it to be 3.

List different activation neurons or functions

● Linear Neuron
● Binary Threshold Neuron
● Stochastic Binary Neuron
● Sigmoid Neuron
● Tanh function
● Rectified Linear Unit (ReLU)

Define Learning rate

The learning rate is a hyper-parameter that controls how much we are adjusting the weights of our network with respect to the loss gradient.

What is Momentum (w.r.t NN optimization)?

Momentum lets the optimization algorithm remembers its last step, and adds some proportion of it to the current step. This way, even if the algorithm is stuck in a flat region, or a small local minimum, it can get out and continue towards the true minimum.

What is the difference between Batch Gradient Descent and Stochastic Gradient Descent?

Batch gradient descent computes the gradient using the whole dataset. This is great for convex or relatively smooth error manifolds. In this case, we move somewhat directly towards an optimum solution, either local or global. Additionally, batch gradient descent, given an annealed learning rate, will eventually find the minimum located in its basin of attraction.
Stochastic gradient descent (SGD) computes the gradient using a single sample. SGD works well (Not well, I suppose, but better than batch gradient descent) for error manifolds that have lots of local maxima/minima. In this case, the somewhat noisier gradient calculated using the reduced number of samples tends to jerk the model out of local minima into a region that hopefully is more optimal.

Epoch vs Batch vs Iteration.

Epoch: one forward pass and one backward pass of all the training examples
Batch: examples processed together in one pass (forward and backward)
Iteration: number of training examples / Batch size

What is the vanishing gradient?

As we add more and more hidden layers, backpropagation becomes less and less useful in passing information to the lower layers. In effect, as information is passed back, the gradients begin to vanish and become small relative to the weights of the networks.

What are dropouts?

Dropout is a simple way to prevent a neural network from overfitting. It is the dropping out of some of the units in a neural network. It is similar to the natural reproduction process, where nature produces offsprings by combining distinct genes (dropping out others) rather than strengthening the co-adapting of them.

What is data augmentation? Can you give some examples?

Data augmentation is a technique for synthesizing new data by modifying existing data in such a way that the target is not changed, or it is changed in a known way. Computer vision is one of the fields where data augmentation is very useful. There are many modifications that we can do to images:
● Resize
● Horizontal or vertical flip
● Rotate, Add noise, Deform
● Modify colors Each problem needs a customized data augmentation pipeline. For example, on OCR, doing flips will change the text and won’t be beneficial; however, resizes and small rotations may help.

What are the components of GAN?

● Generator
● Discriminator

What’s the difference between a generative and discriminative model?

A generative model will learn categories of data while a discriminative model will simply learn the distinction between different categories of data. Discriminative models will generally outperform generative models on classification tasks.

What is Linear Filtering?

Linear filtering is a neighborhood operation, which means that the output of a pixel’s value is decided by the weighted sum of the values of the input pixels.

How can you achieve Blurring through Gaussian Filter?

This is the most common technique for blurring or smoothing an image. This filter improves the resulting pixel found at the center and slowly minimizes the effects as pixels move away from the center. This filter can also help in removing noise in an image.

How can you achieve Blurring through Gaussian Filter?

This is the most common technique for blurring or smoothing an image. This filter improves the resulting pixel found at the center and slowly minimizes the effects as pixels move away from the center. This filter can also help in removing noise in an image.

What is Non-Linear Filtering? How it is used?

Linear filtering is easy to use and implement. In some cases, this method is enough to get the necessary output. However, an increase in performance can be obtained through non-linear filtering. Through non-linear filtering, we can have more control and achieve better results when we encounter a more complex computer vision task.

Explain Median Filtering.

The median filter is an example of a non-linear filtering technique. This technique is commonly used for minimizing the noise in an image. It operates by inspecting the image pixel by pixel and taking the place of each pixel’s value with the value of the neighboring pixel median.
Some techniques in detecting and matching features are:
● Lucas-Kanade
● Harris
● Shi-Tomasi
● SUSAN (smallest uni value segment assimilating nucleus)
● MSER (maximally stable extremal regions)
● SIFT (scale-invariant feature transform)
● HOG (histogram of oriented gradients)
● FAST (features from accelerated segment test)
● SURF (speeded-up robust features)

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Describe the Scale Invariant Feature Transform (SIFT) algorithm

SIFT solves the problem of detecting the corners of an object even if it is scaled. Steps to implement this algorithm:
● Scale-space extrema detection – This step will identify the locations and scales that can still be recognized from different angles or views of the same object in an image.
● Keypoint localization – When possible key points are located, they would be refined to get accurate results. This would result in the elimination of points that are low in contrast or points that have edges that are deficiently localized.
● Orientation assignment – In this step, a consistent orientation is assigned to each key point to attain invariance when the image is being rotated.
● Keypoint matching – In this step, the key points between images are now linked to recognizing their nearest neighbors.

Why Speeded-Up Robust Features (SURF) came into existence?

SURF was introduced to as a speed-up version of SIFT. Though SIFT can detect and describe key points of an object in an image, still this algorithm is slow.

What is Oriented FAST and rotated BRIEF (ORB)?

This algorithm is a great possible substitute for SIFT and SURF, mainly because it performs better in computation and matching. It is a combination of fast key point detector and brief descriptor, which contains a lot of alterations to improve performance. It is also a great alternative in terms of cost because the SIFT and SURF algorithms are patented, which means that you need to buy them for their utilization.

What is image segmentation?

In computer vision, segmentation is the process of extracting pixels in an image that is related.
Segmentation algorithms usually take an image and produce a group of contours (the boundary of an object that has well-defined edges in an image) or a mask where a set of related pixels are assigned to a unique color value to identify it.
Popular image segmentation techniques:
● Active contours
● Level sets
● Graph-based merging
● Mean Shift
● Texture and intervening contour-based normalized cuts

What is the purpose of semantic segmentation?

The purpose of semantic segmentation is to categorize every pixel of an image to a certain class or label. In semantic segmentation, we can see what is the class of a pixel by simply looking directly at the color, but one downside of this is that we cannot identify if two colored masks belong to a certain object.

Explain instance segmentation.

In semantic segmentation, the only thing that matters to us is the class of each pixel. This would somehow lead to a problem that we cannot identify if that class belongs to the same object or not.
Semantic segmentation cannot identify if two objects in an image are separate entities. So to solve this problem, instance segmentation was created. This segmentation can identify two different objects of the same class. For example, if an image has two sheep in it, the sheep will be detected and masked with different colors to differentiate what instance of a class they belong to.

How is panoptic segmentation different from semantic/instance segmentation?

Panoptic segmentation is basically a union of semantic and instance segmentation. In panoptic segmentation, every pixel is classified by a certain class and those pixels that have several instances of a class are also determined. For example, if an image has two cars, these cars will be masked with different colors. These colors represent the same class — car — but point to different instances of a certain class.

Explain the problem of recognition in computer vision.

Recognition is one of the toughest challenges in the concepts in computer vision. Why is recognition hard? For the human eyes, recognizing an object’s features or attributes would be very easy. Humans can recognize multiple objects with very small effort. However, this does not apply to a machine. It would be very hard for a machine to recognize or detect an object because these objects vary. They vary in terms of viewpoints, sizes, or scales. Though these things are still challenges faced by most computer vision systems, they are still making advancements or approaches for solving these daunting tasks.

What is Object Recognition?

Object recognition is used for indicating an object in an image or video. This is a product of machine learning and deep learning algorithms. Object recognition tries to acquire this innate human ability, which is to understand certain features or visual detail of an image.

What is Object Detection and it’s real-life use cases?

Object detection in computer vision refers to the ability of machines to pinpoint the location of an object in an image or video. A lot of companies have been using object detection techniques in their system. They use it for face detection, web images, and security purposes.

Describe Optical Flow, its uses, and assumptions.

Optical flow is the pattern of apparent motion of image objects between two consecutive frames caused by the movement of object or camera. It is a 2D vector field where each vector is a displacement vector showing the movement of points from the first frame to the second
Optical flow has many applications in areas like :
● Structure from Motion
● Video Compression
● Video Stabilization
Optical flow works on several assumptions:
1. The pixel intensities of an object do not change between consecutive frames.
2. Neighboring pixels have similar motion.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is Histogram of Oriented Gradients (HOG)?

HOG stands for Histograms of Oriented Gradients. HOG is a type of “feature descriptor”. The intent of a feature descriptor is to generalize the object in such a way that the same object (in this case a person) produces as close as possible to the same feature descriptor when viewed under different conditions. This makes the classification task easier.

What’s the difference between valid and same padding in a CNN?

This question has more chances of being a follow-up question to the previous one. Or if you have explained how you used CNNs in a computer vision task, the interviewer might ask this question along with the details of the padding parameters.
● Valid Padding: When we do not use any padding. The resultant matrix after convolution will have dimensions (n – f + 1) X (n – f + 1)
● Same padding: Adding padded elements all around the edges such that the output matrix will have the same dimensions as that of the input matrix

What is BOV: Bag-of-visual-words (BOV)?

BOV also called the bag of key points, is based on vector quantization. Similar to HOG features, BOV features are histograms that count the number of occurrences of certain patterns within a patch of the image.

What is Poselets? Where are poselets used?

Poselets rely on manually added extra keypoints such as “right shoulder”, “left shoulder”, “right knee” and “left knee”. They were originally used for human pose estimation

Explain Textons in context of CNNs

A texton is the minimal building block of vision. The computer vision literature does not give a strict definition for textons, but edge detectors could be one example. One might argue that deep learning techniques with Convolution Neuronal Networks (CNNs) learn textons in the first filters.

What is Markov Random Fields (MRFs)?

MRFs are undirected probabilistic graphical models which are a wide-spread model in computer vision. The overall idea of MRFs is to assign a random variable for each feature and a random variable for each pixel.

Explain the concept of superpixel?

A superpixel is an image patch that is better aligned with intensity edges than a rectangular patch.
Superpixels can be extracted with any segmentation algorithm, however, most of them produce highly irregular superpixels, with widely varying sizes and shapes. A more regular space tessellation may be desired.

What is Non-maximum suppression(NMS) and where is it used?

NMS is often used along with edge detection algorithms. The image is scanned along the image gradient direction, and if pixels are not part of the local maxima they are set to zero. It is widely used in object detection algorithms.

Describe the use of Computer Vision in Healthcare.

Computer vision has also been an important part of advances in health-tech. Computer vision algorithms can help automate tasks such as detecting cancerous moles in skin images or finding symptoms in x-ray and MRI scans

Describe the use of Computer Vision in Augmented Reality & Mixed Reality

Computer vision also plays an important role in augmented and mixed reality, the technology that enables computing devices such as smartphones, tablets, and smart glasses to overlay and embed virtual objects on real-world imagery. Using computer vision, AR gear detects objects in the real world in order to determine the locations on a device’s display to place a virtual object.
For instance, computer vision algorithms can help AR applications detect planes such as tabletops, walls, and floors, a very important part of establishing depth and dimensions and placing virtual objects in the physical world.

Describe the use of Computer Vision in Facial Recognition

Computer vision also plays an important role in facial recognition applications, the technology that enables computers to match images of people’s faces to their identities. Computer vision algorithms detect facial features in images and compare them with databases of face profiles.
Consumer devices use facial recognition to authenticate the identities of their owners. Social media apps use facial recognition to detect and tag users. Law enforcement agencies also rely on facial recognition technology to identify criminals in video feeds.

Describe the use of Computer Vision in Self-Driving Cars

Computer vision enables self-driving cars to make sense of their surroundings. Cameras capture video from different angles around the car and feed it to computer vision software, which then processes the images in real-time to find the extremities of roads, read traffic signs, detect other cars, objects, and pedestrians. The self-driving car can then steer its way on streets and highways, avoid hitting obstacles, and (hopefully) safely drive its passengers to their destination.

Explain famous Computer Vision tasks using a single image example.

Many popular computer vision applications involve trying to recognize things in photographs; for example:
Object Classification: What broad category of object is in this photograph?
Object Identification: Which type of a given object is in this photograph?
Object Verification: Is the object in the photograph?
Object Detection: Where are the objects in the photograph?
Object Landmark Detection: What are the key points for the object in the photograph?
Object Segmentation: What pixels belong to the object in the image?
Object Recognition: What objects are in this photograph and where are they?

Explain the distinction between Computer Vision and Image Processing.

Computer vision is distinct from image processing.
Image processing is the process of creating a new image from an existing image, typically simplifying or enhancing the content in some way. It is a type of digital signal processing and is not concerned with understanding the content of an image.
A given computer vision system may require image processing to be applied to raw input, e.g. pre-processing images.
Examples of image processing include:
● Normalizing photometric properties of the image, such as brightness or color.
● Cropping the bounds of the image, such as centering an object in a photograph.
● Removing digital noise from an image, such as digital artifacts from low light levels

Explain business use cases in computer vision.

● Optical character recognition (OCR)
● Machine inspection
● Retail (e.g. automated checkouts)
● 3D model building (photogrammetry)
● Medical imaging
● Automotive safety
● Match move (e.g. merging CGI with live actors in movies)
● Motion capture (mocap)
● Surveillance
● Fingerprint recognition and biometrics

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

What is the Boltzmann Machine?

One of the most basic Deep Learning models is a Boltzmann Machine, resembling a simplified version of the Multi-Layer Perceptron. This model features a visible input layer and a hidden layer — just a two-layer neural net that makes stochastic decisions as to whether a neuron should be on or off. Nodes are connected across layers, but no two nodes of the same layer are connected.

What Is the Role of Activation Functions in a Neural Network?

At the most basic level, an activation function decides whether a neuron should be fired or not. It accepts the weighted sum of the inputs and bias as input to any activation function. Step function,
Sigmoid, ReLU, Tanh, and Softmax are examples of activation functions.

What Is the Difference Between a Feedforward Neural Network and Recurrent Neural Network?

A Feedforward Neural Network signals travel in one direction from input to output. There are no feedback loops; the network considers only the current input. It cannot memorize previous inputs (e.g., CNN).

What Are the Applications of a Recurrent Neural Network (RNN)?

The RNN can be used for sentiment analysis, text mining, and image captioning. Recurrent Neural Networks can also address time series problems such as predicting the prices of stocks in a month or quarter.

What Are the Softmax and ReLU Functions?

Softmax is an activation function that generates the output between zero and one. It divides each output, such that the total sum of the outputs is equal to one. Softmax is often used for output layers.

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Machine Learning For Dummies
Machine Learning For Dummies

Machine Learning Techniques

What Is Overfitting, and How Can You Avoid It?

Overfitting is a situation that occurs when a model learns the training set too well, taking up random fluctuations in the training data as concepts. These impact the model’s ability to generalize and don’t apply to new data.
When a model is given the training data, it shows 100 percent accuracy—technically a slight loss. But, when we use the test data, there may be an error and low efficiency. This condition is known as overfitting.
There are multiple ways of avoiding overfitting, such as:
● Regularization. It involves a cost term for the features involved with the objective function
● Making a simple model. With lesser variables and parameters, the variance can be reduced
● Cross-validation methods like k-folds can also be used
● If some model parameters are likely to cause overfitting, techniques for regularization like LASSO can be used that penalize these parameters

What is meant by ‘Training set’ and ‘Test Set’?

We split the given data set into two different sections namely, ‘Training set’ and ‘Test Set’.
‘Training set’ is the portion of the dataset used to train the model.
‘Testing set’ is the portion of the dataset used to test the trained model.

How Do You Handle Missing or Corrupted Data in a Dataset?

One of the easiest ways to handle missing or corrupted data is to drop those rows or columns or replace them entirely with some other value.
There are two useful methods in Pandas:
● IsNull() and dropna() will help to find the columns/rows with missing data and drop them
● Fillna() will replace the wrong values with a placeholder value

How Do You Design an Email Spam Filter?

Building a spam filter involves the following process:

● The email spam filter will be fed with thousands of emails
● Each of these emails already has a label: ‘spam’ or ‘not spam.’
● The supervised machine learning algorithm will then determine which type of emails are being marked as spam based on spam words like the lottery, free offer, no money, full refund, etc.
● The next time an email is about to hit your inbox, the spam filter will use statistical analysis and algorithms like Decision Trees and SVM to determine how likely the email is spam
● If the likelihood is high, it will label it as spam, and the email won’t hit your inbox
● Based on the accuracy of each model, we will use the algorithm with the highest accuracy after testing all the models

Explain bagging.

Bagging, or Bootstrap Aggregating, is an ensemble method in which the dataset is first divided into multiple subsets through resampling.
Then, each subset is used to train a model, and the final predictions are made through voting or averaging the component models.
Bagging is performed in parallel.

What is the ROC Curve and what is AUC (a.k.a. AUROC)?

The ROC (receiver operating characteristic) the performance plot for binary classifiers of True Positive Rate (y-axis) vs. False Positive Rate (xaxis).
AUC is the area under the ROC curve, and it’s a common performance metric for evaluating binary classification models.
It’s equivalent to the expected probability that a uniformly drawn random positive is ranked before a uniformly drawn random negative.

 

What are the various Machine Learning algorithms?

 

 

What is cross-validation?

 

Reference: k-fold cross validation 

 

Cross-validation is a resampling procedure used to evaluate machine learning models on a limited data sample. The procedure has a single parameter called k that refers to the number of groups that a given data sample is to be split into. As such, the procedure is often called k-fold cross-validation. When a specific value for k is chosen, it may be used in place of k in the reference to the model, such as k=10 becoming 10-fold cross-validation. Mainly used in backgrounds where the objective is forecast, and one wants to estimate how accurately a model will accomplish in practice.

 

Cross-validation is primarily used in applied machine learning to estimate the skill of a machine learning model on unseen data. That is, to use a limited sample in order to estimate how the model is expected to perform in general when used to make predictions on data not used during the training of the model.

 

It is a popular method because it is simple to understand and because it generally results in a less biased or less optimistic estimate of the model skill than other methods, such as a simple train/test split.

 

The general procedure is as follows:
1. Shuffle the dataset randomly.
2. Split the dataset into k groups
3. For each unique group:
a. Take the group as a hold out or test data set
b. Take the remaining groups as a training data set
c. Fit a model on the training set and evaluate it on the test set
d. Retain the evaluation score and discard the model
4. Summarize the skill of the model using the sample of model evaluation scores

What are 3 data preprocessing techniques to handle outliers?

1. Winsorize (cap at threshold).
2. Transform to reduce skew (using Box-Cox or similar).
3. Remove outliers if you’re certain they are anomalies or measurement errors.

How much data should you allocate for your training, validation, and test sets?

You have to find a balance, and there’s no right answer for every problem.
If your test set is too small, you’ll have an unreliable estimation of model performance (performance statistic will have high variance). If your training set is too small, your actual model parameters will have a high variance.
A good rule of thumb is to use an 80/20 train/test split. Then, your train set can be further split into train/validation or into partitions for cross-validation.

What Is a False Positive and False Negative and How Are They Significant?

False positives are those cases which wrongly get classified as True but are False.
False negatives are those cases which wrongly get classified as False but are True.
In the term ‘False Positive’, the word ‘Positive’ refers to the ‘Yes’ row of the predicted value in
the confusion matrix. The complete term indicates that the system has predicted it as a positive, but the actual value is negative.

What’s a Fourier transform?

A Fourier transform is a generic method to decompose generic functions into a superposition of symmetric functions. Or as this more intuitive tutorial puts it, given a smoothie, it’s how we find the recipe. The Fourier transform finds the set of cycle speeds, amplitudes, and phases to match any time signal. A Fourier transform converts a signal from time to frequency domain — it’s a very common way to extract features from audio signals or other time series such as sensor data.

 

Machine Learning Cheat Sheets, Tutorial, Practical examples, References, Datasets

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Machine Learning Cheat Sheet

Machine Learning Cheat Sheets

Download it here

Credit: Remi Canard

TensorFlow Practical Examples and Tutorial

– Basic Models
Linear Regression
Logistic Regression
Word2Vec (Word Embedding)

– Neural Networks
Simple Neural Network
Convolutional Neural Network
Recurrent Neural Network (LSTM)
Bi-directional Recurrent Neural Network (LSTM)
Dynamic Recurrent Neural Network (LSTM)

-Unsupervised
Auto-Encoder
DCGAN (Deep Convolutional Generative Adversarial Networks)

-Utilities:
Save and Restore a model
Build Custom Layers & Modules

– Data Management
Load and Parse data
Build and Load TFRecords
Image Transformation (i.e. Image Augmentation)

TensorFlow Examples abd Tutorials

Download it here

Credit: Alex Wang

Cool MLOps repository of free talks, books, papers and more

Link to the repo:

Image preview

Machine Learning  Training Videos

 



References

1 https://springboard.com
2 https://simplilearn.com
3 https://geeksforgeeks.org
4 https://elitedatascience.com
5 https://analyticsvidhya.com
6 https://guru99.com
7 https://intellipaat.com
8 https://towardsdatascience.com
9 https://mygreatlearning.com
10 https://mindmajix.com
11 https://toptal.com
12 https://glassdoor.co.in
13 https://udacity.com
14 https://educba.com
15 https://analyticsindiamag.com
16 https://ubuntupit.com
17 https://javatpoint.com
18 https://quora.com
19 hackr.io
20 kaggle.com
21 https://www.linkedin.com/in/stevenouri/

 

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Explain differences between AI, Machine Learning and NLP

 

 

 

 

 

 

 

 

 

 

 

 

 

Artificial IntelligenceMachine LearningNatural Language Processing
It is the technique t create smarter machinesMachine Learning is the term used for systems that learn from experienceThis is the set of system that has the ability to understand the language
AI includes human interventionMachine Learning purely involves the working of computers and no human interventionNLP links both computer and human languages
Artificial intelligence is a broader concept than Machine LearningML is a narrow concept and is a subset of AI 

Top Machine Learning Algorithms for Predictions:

Top ML algorithms for predictions

TensorFlow Interview Questions and Answers

Tensorflow Interview Questions and Answers

Direct link here

Machine Learning For Dummies  on iOs

Machine Learning For Dummies on Windows

Machine Learning For Dummies Web/Android 

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Machine Learning For Dummies
Machine Learning For Dummies

Machine learning is just one component of a larger field called artificial intelligence (AI). AI researchers have done an excellent job at describing the fundamental problems they must solve to achieve intelligent behavior; these problems fall into four general categories: representation, reasoning, learning, and search.

Basically, all of AI research can be classified under these headings; for example, language understanding is a special case of representation (natural language), planning is a special case of reasoning (analogical logical inferences), learning to play chess is a special case of learning (policy search in the game tree), and table lookup is a special case of search (symbol-table lookups). We will focus on two: representation and search.

What follows are our ten favorite problems/areas for the next decade or so. Each one has been researched quite heavily already, but we think that there are no silver bullets yet discovered nor are there any obvious candidates lurking in the wings waiting to take over. Each area has a different flavor to it; all have something to offer the machine learning community, and we believe that many will find fertile ground for their own investigations.

Machine learning methods are useful on large problems, which is becoming increasingly important as applications such as speech recognition are moving into real-world situations outside the lab (e.g., using voice commands while driving). Solution: This is a difficult one because there are many possible solutions to this problem; all will require advances in both theoretical and experimental techniques but we do not know what they are yet. A better understanding of why certain learning algorithms work well on some types of problems but not others may provide insights into how to scale them up. Some examples of the types of problems we would like to tackle include: (i) learning from large databases, (ii) learning in multiple domains, and (iii) learning task-specific knowledge.

Artificial intelligence methods have been used to solve combinatorial problems such as chess playing and problem-solving; these are problems that can be represented as a search tree using nodes representing possible moves for each player. These methods work well on small problems but often fail when applied to larger real-world problems because there are too many options in the search trees that must be explored. For example, consider a game where there are 100 moves per second for each player with 10^100 different games possible over a 40 year lifetime. Solving the AI problem amounts to finding a winning strategy. This is much different from the type of problems we are used to solving which normally fit in memory and where the number of potential options can be kept manageable. Solution: We need better methods than those currently available for searching through very large trees; these could involve ideas from machine learning, such as neural networks or evolutionary algorithms.

Searching for solutions to a problem among all possible alternatives is an important capability but one that has not been researched nearly enough due to its complexity. A brute-force search would seem to require enumerating all alternatives, which is impossible even on extremely simple problems, whereas other approaches seem so specialized that they have little value outside their specific domain (and sometimes not even there). In contrast, machine learning methods can be applied to virtually any problem where the solution space is finite (e.g., finding a path through a graph or board games like chess).

The brute-force approach of enumerating all possible combinations has been successfully applied to optimization problems where only a few desirable solutions are available, but there are many applications that require solving very large problems with thousands or millions of potential solutions. Examples include the Traveling Salesman Problem and scheduling tasks for an airline crew using dozens of variables (e.g., number of passengers flying, weight, the distance between origin and destination cities), a task which becomes more difficult because it must deal with occasional breakdowns in equipment. Any feasible algorithm will require shortcuts that often involve approximations or heuristics. Source.

What is the main purpose of using PCA on a dataset, and what are some examples of its application?

PCA is short for Principal Component Analysis, and it’s a technique used to reduce the dimensionality of a dataset. In other words, it helps you to find the important Variables in a dataset and get rid of the noise. PCA is used in a variety of fields, from image recognition to facial recognition to machine learning.

PCA has a few main applications:
– Reducing the number of features in a dataset
– Finding relationships between features
– Identifying clusters in data
– Visualizing data

Let’s take a look at an example. Say you have a dataset with 1000 features (variables). PCA can help you reduce that down to, say, 10 features that explain the majority of variance in the data. This is helpful because it means you can build a model with far fewer features, which makes it simpler and faster. In addition, PCA can help you to find relationships between features and identify clusters in data. All of this can be extremely helpful in understanding and using your data.

PCA is an important tool in Machine Learning, and has a number of applications. The main purpose of PCA is to reduce the dimensionality of a dataset, while still retaining as much information as possible. This can be useful when dealing with very large datasets, as it can make training and testing faster and more efficient. PCA is also often used for data visualization, as it can help to create clear and concise visualizations of high-dimensional data. Finally, PCA can be used for feature selection, as it can help to identify the most important features in a dataset. PCA is a powerful tool that can be applied in many different ways, and is an essential part of any Machine Learning workflow.

What are subservient sounding male names suitable for an automated assistant?

Artificial intelligence is increasingly becoming a staple in our lives, with everything from our homes to our workplaces being automated to some degree. And as AI becomes more ubiquitous, we are starting to see a trend of subservient-sounding names being given to male automated assistants. This is likely due to a combination of factors, including the fact that women are still primarily seen as domestic servants and the fact that many people find it easier to relate to a male voice. Whatever the reason, it seems that subservient-sounding names are here to stay when it comes to male AI. So if you’re looking for a name for your new automated assistant, here are some subservient-sounding male names to choose from:

– Jasper: A popular name meaning “treasurer” or “bringer of riches.”
– Custer: A name derived from the Latin word for “servant.”
– Luther: A Germanic name meaning “army of warriors.”
– Benson: A name of English origin meaning “son of Ben.”
– Wilfred: A name of Germanic origin meaning “desires peace.”

In recent years, there has been an increasing trend of using subservient sounding male names for automated assistants. Artificial intelligence is becoming more prevalent in our everyday lives, and automation is slowly but surely taking over many routine tasks. As such, it’s no surprise that we’re seeing a name trend emerge that reflects our growing dependence on these technologies. So what are some suitable names for an automated assistant? How about “Robo-Bob”? Or “Mecha-Mike”? Perhaps even “Cyber-Steve”? Whatever you choose, just be sure to pick a name that sounds suitably subservient! After all, your automated assistant should reflect your growing dependency on technology… and not your growing dominance over it!

How do you calculate user churn rate?

Churn rate is a metric that measures the percentage of users who leave or discontinue using a service within a given time period. The churn rate is an important metric for businesses to track because it can help them identify areas where their product or service is losing users. There are many ways to calculate the churn rate, but one of the most popular methods is to use machine learning or artificial intelligence. Artificial intelligence can help identify patterns in user behavior that may indicate that someone is about to leave the service. By tracking these patterns, businesses can be proactive in addressing user needs and reducing the chances of losing them. In addition, automation can also help reduce the churn rate by making it easier for users to stay with the service. Automation can handle tasks like customer support and billing, freeing up users’ time and making it less likely that they will discontinue their subscription. By using machine learning and artificial intelligence, businesses can more accurately predict and prevent user churn.

There are a few different ways to calculate the user churn rate using artificial intelligence. One way is to use a technique called Artificial Neural Networks. This involves training a computer to recognize patterns in data. Once the computer has learned to recognize these patterns, it can then make predictions about future data. Another way to calculate the user churn rate is to use a technique called Support Vector Machines. This approach uses algorithms to find the boundaries between different groups of data. Once these boundaries have been found, the algorithm can then make predictions about new data points. Finally, there is a technique called Bayesian inference. This approach uses probability theory to make predictions about future events. By using these three techniques, it is possible to calculate the user churn rate with a high degree of accuracy.

 

#datascience #machinelearning

Machine Learning Techniques illustrated

How to confuse Machine Learning and AI?

Folks with no educational background taking a MOOC or two in deep learning, entering the field, and skipping over basic concepts in machine learning–specificity/sensitivity, the difference between supervised and unsupervised learning, linear regression, ensembles, proper design of a study/test, probability distributions… With enough MOOCs, you can sound like you know what you are doing, but as soon as something goes wrong or changes slightly, there’s no knowledge about how to fix it. Big problem in employment, particularly when hiring a first machine learning engineer/data scientist.. Source: Colleen Farrelly

With rapid developments of artificial intelligence (AI) technology, the use of AI technology to mine clinical data has become a major trend in medical industry. Utilizing advanced AI algorithms for medical image analysis, one of the critical parts of clinical diagnosis and decision-making, has become an active research area both in industry and academia. Recent applications of deep leaning in medical image analysis involve various computer vision-related tasks such as classification, detection, segmentation, and registration. Among them, classification, detection, and segmentation are fundamental and the most widely used tasks that can be done with Scale but the rest of the more demanding methods require a more sophisticated platform for example Tasq.

Although there exist a number of reviews on deep learning methods on medical image analysis, most of them emphasize either on general deep learning techniques or on specific clinical applications. The most comprehensive review paper is the work of Litjens et al. published in 2017. Deep learning is such a quickly evolving research field; numerous state-of-the-art works have been proposed since then.

AI Technologies in Medical Image Analysis

Different medical imaging modalities have their unique characteristics and different responses to human body structure and organ tissue and can be used in different clinical purposes. The commonly used image modalities for diagnostic analysis in clinic include projection imaging (such as X-ray imaging), computed tomography (CT), ultrasound imaging, and magnetic resonance imaging (MRI). MRI sequences include T1, T1-w, T2, T2-w, diffusion-weighted imaging (DWI), apparent diffusion coefficient (ADC), and fluid attenuation inversion recovery (FLAIR). Figure 1 demonstrates a few examples of medical image modalities and their corresponding clinical applications.

Image Classification for Medical Image Analysis

As a fundamental task in computer vision, image classification plays an essential role in computer-aided diagnosis. A straightforward use of image classification for medical image analysis is to classify an input image or a series of images as either containing one (or a few) of predefined diseases or free of diseases (i.e., healthy case). Typical clinical applications of image classification tasks include skin disease identification in dermatology, eye disease recognition in ophthalmology (such as diabetic retinopathy, glaucoma, and corneal diseases). Classification of pathological images for various cancers such as breast cancer and brain cancer also belongs to this area.

Convolutional neural network (CNN) is the dominant classification framework for image analysis. With the development of deep learning, the framework of CNN has continuously improved. AlexNet was a pioneer convolutional neural network, which was composed of repeated convolutions, each followed by ReLU and max pooling operation with stride for downsampling. The proposed VGGNet used convolution kernels and maximum pooling to simplify the structure of AlexNet and showed improved performance by simply increasing the number and depth of the network. Via combining and stacking , and convolution kernels and pooling, the inception network and its variants increased the width and the adaptability of the network. ResNet and DenseNet both used skip connections to relieve the gradient vanishing. SENet proposed a squeeze-and-excitation module which enabled the model to pay more attention to the most informative channel features. The family of EfficientNet applied AUTOML and a compound scaling method to uniformly scale the width, depth, and resolution of the network in a principled way, resulting in improved accuracy and efficiency. Source: Kelly Holland

GPUs also process things. It’s just that they’re better and faster at “specific” things.

The main stuff a GPU is “awesome” at, exactly because it is designed to be specific with those: Matrix maths. The sorts of calculation used when converting a bunch of 3d points (XYZ values) into an approximation of how such a shape would look from a camera. I.e. rendering a 2d picture from a 3d object – exactly why a GPU is made in the first place: https://www.3dgep.com/3d-math-primer-for-game-programmers-matrices/

The sorts of calculations used in current “AI” ? Guess what? Matrix maths:

https://rossbulat.medium.com/ai-essentials-working-with-matrices-2ceb9ca3bd1b

By Irne Barnard

This is perhaps the most important question in computational learning theory. In fact, some of the most important theorems of machine learning like the No Free Lunch Theorem and the Fundamental Theorem of Statistical Learning are aimed at answering this very question.

Formally, the smallest number of data points needed for successfully learning a classification rule using a machine learning (ML) algorithm is called the sample complexity of the algorithm. Now, you might wonder why sample complexity is such a big deal. It’s because sample complexity is to ML algorithms what computational complexity is to any algorithm. It measures the minimum amount of resource (i.e. the data) that is required to achieve the desired goal.

There are several interesting answers to the question of sample complexity, that arise from various assumptions on the learner. In what follows, I will give the answer under some popular assumptions/scenarios.

Scenario 1: Perfect Learning

In our first scenario, we consider the problem of learning the correct hypothesis (classification rule) amongst a set of plausible hypotheses. The data is sampled independently from an unknown probability distribution.

It turns out that under no further assumptions on the data-generating probability distribution, the problem is impossible. In other words, there is no algorithm that can learn the correct classification rule perfectly from any finite amount of data. This result is called the No Free Lunch Theorem in machine learning. I’ve discuss this result in more detail here.

Scenario 2: Probably Approximately Correct (PAC) Learning

For the second scenario, we consider the problem of learning the correct hypothesis approximately, with high probability. That is, our algorithm may fail to identify even an approximately correct hypothesis with some small probability. This relaxation allows us to give a slightly more useful answer to the question.

The answer to this question is of the order of the VC-dimension of the hypothesis class. More precisely, if we want the algorithm to be approximately correct with an error of at most ϵϵ with a probability of at least 1δ1−δ, then we need a minimum of dϵlog(1ϵδ)dϵlog⁡(1ϵδ), where dd is the VC-dimension of the hypothesis class. Note that dd can be infinite for certain hypothesis classes. In that case, it is not possible to succeed in the learning task even approximately, even with high probability. On the other hand, if dd is finite, we say that the hypothesis class is (ϵ,δ)(ϵ,δ)−PAC learnable. (I explain PAC-learnability in more detail in this answer.)

Scenario 3: Learning with a Teacher

In the previous two scenarios, we assume that the data that is presented to the learner is randomly sampled from an unknown probability distribution. For this scenario, we do away with the randomness. Instead, we assume that the learner is presented with a carefully chosen set of training data points that are picked by a benevolent teacher. (By benevolent teacher, I mean that the teacher tries to make the learner guess the correct hypothesis with the fewest number of data points.)

In this case, the answer to the question is the teaching dimension. It is interesting to note that there is no straightforward relation between the teaching dimension and VC-dimension of a hypothesis class. They can be arbitrarily far from each other. (If you’re curious to know the relation between the two, here is a nice paper.)


In addition to these, there are other notions of “dimension” that characterize the sample complexity of a learning task under different scenarios. For example, there is the Littlestone dimension for online learning and Natarajan dimension for multi-class learning. Intuitively, these dimensions capture the inherent hardness of a machine learning task. The harder the task, the higher the dimension and the corresponding sample complexity.


To those of you seeking for exact numbers, here’s a note I added in the comments section: I wish I could add some useful empirical results, but the sample complexity bounds obtained by the PAC-learning approach are really loose to the point of being useless in case of most state-of-the-art ML algorithms like deep learning. So, the results I presented are basically a theoretical curiosity at this point. However, this might change in the near future as lots of researchers are working on strengthening this framework.

Source: Muni Sreenivas Pydi

As mentioned in the other answer, this can be understood using the concept of bias-variance tradeoff.

For any machine learning model, want to find a function that approximately fits your data. So, you essentially define the following:

  • Class of functions : Instead of searching in the space of all possible functions, you restrict the space of functions that the algorithm searches over. For example, a linear classifier will search among all possible lines, but will not consider more complex curves.
  • Loss function : This is used to compare two functions from the above class of functions. For instance, in SVM, you would prefer line 1 to line 2 if line 1 has a larger margin than line 2.

Now, the simpler your class of functions is, the smaller the amount of data required. To get some intuition for this, think about a regression problem that has three features. So, a linear function class will have the following form:

y=a0+a1x1+a2x2+a3x3y=a0+a1x1+a2x2+a3x3

Every point (p, q, r, s) in the 4-dimensional space corresponds to a function of the above form, namely y=p+qx1+rx2+sx3y=p+qx1+rx2+sx3. So, you need to find one point in that 4D space that fits your data well.

Now, if instead of the class of linear functions, you chose quadratic functions, your functions would be of the following form:

y=a0+a1x1+a2x2+ a3x3+a4x1x2+ a5x2x3+a6x1x3+ a7x21+a8x22+a9x23y = a0+a1x1+a2x2+a3x3 + a4x1x2+a5x2x3+a6x1x3+a7x12+a8x22+a9x32

So now, you have to search for the best point in a 10D space! Therefore, you need more data to distinguish these larger number of points from each other.

With that intuition, we can say that to learn from small amount of data, you want to define a small enough function class.

Note: While in the above example, we simply look at the no. of parameters to get a sense of complexity of the function class, in general, more parameters does not necessarily mean more complexity [for instance, if a lot of the parameters are strongly correlated].

Source: Prasoon Goyal

Best Machine Learning Books That All Data Scientists Must Read.

best ml books

1. Artificial Intelligence: A Modern Approach

Artificial Intelligence: A Modern Approach
Artificial Intelligence: A Modern Approach

Experts Opinions

I like this book very much. When in doubt I look there, and usually find what I am looking for, or I find references on where to go to study the problem more in depth. I like that it tries to show how various topics are interrelated, and to give general architectures for general problems … It is a jump in quality with respect to the AI books that were previously available. — Prof. Giorgio Ingargiola (Temple).

Really excellent on the whole and it makes teaching AI a lot easier. — Prof. Ram Nevatia (USC).

It is an impressive book, which begins just the way I want to teach, with a discussion of agents, and ties all the topics together in a beautiful way. — Prof. George Bekey (USC). Buy it now

2. Deep Learning (Adaptive Computation and Machine Learning series)

Experts Opinions

“Written by three experts in the field, Deep Learning is the only comprehensive book on the subject.” —Elon Musk, cochair of OpenAI; cofounder and CEO of Tesla and SpaceX.

“If you want to know here deep learning came from, what it is good for, and where it is going, read this book.” —Geoffrey Hinton FRS, Professor, University of Toronto, Research Scientist at Google. Buy it

3. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems 2nd Edition

Experts Opinions

“An exceptional resource to study Machine Learning. You will find clear-minded, intuitive explanations, and a wealth of practical tips.” —François Chollet, Author of Keras, author of Deep Learning with Python.

“This book is a great introduction to the theory and practice of solving problems with neural networks; I recommend it to anyone interested in learning about practical ML.” — Peter Warden, Mobile Lead for TensorFlow. Buy it.

4. Python Machine Learning – Second Edition: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2nd Edition

 

First things first, I don’t think there are many questions of the form “Is it a good practice to always X in machine learning” where the answer is going to be definitive. Always? Always always? Across parametric, non-parametric, Bayesian, Monte Carlo, social science, purely mathematic, and million feature models? That’d be nice, wouldn’t it! Anyway feel free to check out this interactive demo from deepchecks.

Concretely though, here are a few ways in which: it just depends.

Some times when normalizing is good:

1) Several algorithms, in particular SVMs come to mind, can sometimes converge far faster on normalized data (although why, precisely, I can’t recall).

2) When your model is sensitive to magnitude, and the units of two different features are different, and arbitrary. This is like the case you suggest, in which something gets more influence than it should.

But of course — not all algorithms are sensitive to magnitude in the way you suggest. Linear regression coefficients will be identical if you do, or don’t, scale your data, because it’s looking at proportional relationships between them.

Some times when normalizing is bad:

1) When you want to interpret your coefficients, and they don’t normalize well. Regression on something like dollars gives you a meaningful outcome. Regression on proportion-of-maximum-dollars-in-sample might not.

2) When, in fact, the units on your features are meaningful, and distance does make a difference! Back to SVMs — if you’re trying to find a max-margin classifier, then the units that go into that ‘max’ matter. Scaling features for clustering algorithms can substantially change the outcome. Imagine four clusters around the origin, each one in a different quadrant, all nicely scaled. Now, imagine the y-axis being stretched to ten times the length of the the x-axis. instead of four little quadrant-clusters, you’re going to get the long squashed baguette of data chopped into four pieces along its length! (And, the important part is, you might prefer either of these!)

In I’m sure unsatisfying summary, the most general answer is that you need to ask yourself seriously what makes sense with the data, and model, you’re using. Source: ABC of Data Science and ML

How do you prepare data for XGBoost?

Data preparation is a critical step in the data science process, and it is especially important when working with XGBoost. XGBoost is a powerful machine learning algorithm that can provide accurate predictions on data sets of all sizes. However, in order to get the most out of XGBoost, it is important to prepare the data in a way that is conducive to machine learning. This means ensuring that the data is clean, feature engineering has been performed, and that the data is in a format that can be easily consumed by the algorithm. By taking the time to prepare the data properly, data scientists can significantly improve the performance of their machine learning models.

When preparing the dataset for your machine learning model, you should use one-hot encoding on what type of data?

In machine learning and data science, one-hot encoding is a process by which categorical data is converted into a format that is suitable for use with machine learning algorithms. The categorical data is first grouped by type, and then a binary value is assigned to each group. This binary value corresponds to the group’s position in the encoding scheme. For example, if there are three groups, the first group would be assigned a value of ‘0’, the second group would be assigned a value of ‘1’, and the third group would be assigned a value of ‘2’. One-hot encoding is often used when working with categorical data, as it can help to improve the performance of machine learning models. In addition, one-hot encoding can also make it easier to visualize the relationship between different categories.

In machine learning and data science, one-hot encoding is a method used to convert categorical features into numerical features. This is often necessary when working with machine learning models, as many models can only accept numerical input. However, one-hot encoding is not without its problems. The most significant issue is the potential for increased dimensionality – if a dataset has too many features, it can be difficult for the model to learn from the data. In addition, one-hot encoding can create sparse datasets, which can also be difficult for some machine learning models to handle. Despite these issues, one-hot encoding remains a popular method for preparing data for machine learning models.

A retail company wants to start personalizing product recommendations to visitors of their website. They have historical data of what products the users have purchased and want to implement the system for new users, prior to them purchasing a product. What’s one way of phrasing a machine learning problem for this situation?

For this retail company, a machine learning problem could be phrased as a prediction problem. The goal would be to build a model that can take in data about a new user (such as demographic information and web browsing history) and predict which products they are likely to purchase. This would allow the company to give each new user personalized product recommendations, increasing the chances of making a sale. Data science techniques such as feature engineering and model selection would be used to build the best possible prediction model. By phrasing the machine learning problem in this way, the retail company can make the most of their historical data and improve the user experience on their website.

There are many ways to frame a machine learning problem for a retail company that wants to start personalizing product recommendations to visitors of their website. One way is to focus on prediction: using historical data of what products users have purchased, can we predict which products new users will be interested in? This is a task that machine learning is well suited for, and with enough data, we can build a model that accurately predicts product interests for new users. Another way to frame the problem is in terms of classification: given data on past purchases, can we classify new users into groups based on their product interests? This would allow the retail company to more effectively target personalization efforts. There are many other ways to frame the machine learning problem, depending on the specific goals of the company. But no matter how it’s framed, machine learning can be a powerful tool for personalizing product recommendations.

A data scientist is trying to determine how a model is doing based on training evaluation. The train accuracy plateaus out at around 70% and the validation accuracy is 67%. How should the data scientist interpret these results?

When working with machine learning models, it is important to evaluate how well the model is performing. This can be done by looking at the train and validation accuracy. In this case, the train accuracy has plateaued at around 70% and the validation accuracy is 67%. There are a few possible explanations for this. One possibility is that the model is overfitting on the training data. This means that the model is able to accurately predict labels for the training data, but it is not as effective at generalizing to new data. Another possibility is that there is a difference in the distribution of the training and validation data. If the validation data is different from the training data, then it makes sense that the model would have a lower accuracy on the validation data. To determine which of these explanations is most likely, the data scientist should look at the confusion matrix and compare the results of the training and validation sets. If there are large differences between the two sets, then it is likely that either overfitting or a difference in distributions is to blame. However, if there isn’t a large difference between the sets, then it’s possible that 70% is simply the best accuracy that can be achieved given the data.

One important consideration in machine learning is how well a model is performing. This can be determined in a number of ways, but one common method is to split the data into a training set and a validation set. The model is then trained on the training data and evaluated on the validation data. If the model is performing well, we would expect to see a similar accuracy on both the training and validation sets. However, in this case the training accuracy plateaus out at around 70% while the validation accuracy is only 67%. This could be indicative of overfitting, where the model has fit the training data too closely and does not generalize well to new data. In this case, the data scientist should look for ways to improve the model so that it performs better on the validation set.

When updating your weights using the loss function, what dictates how much change the weights should have?

In machine learning and data science, the learning rate is a parameter that dictates how much change the weights should have when updating them using the loss function. The learning rate is typically a small value between 0 and 1. A higher learning rate means that the weights are updated more quickly, which can lead to faster convergence but can also lead to instability. A lower learning rate means that the weights are updated more slowly, which can lead to slower convergence but can also help avoid overfitting. The optimal learning rate for a given problem can be found through trial and error. The bias term is another parameter that can affect the weight updates. The bias term is used to prevent overfitting by penalizing models that make too many assumptions about the data. The initial weights are also important, as they determine where the model starts on the optimization landscape. The batch size is another important parameter, as it defines how many training examples are used in each iteration of weight updates. A larger batch size can lead to faster convergence, but a smaller batch size can help avoid overfitting. Finding the optimal values for all of these parameters can be a challenge, but doing so is essential for training high-quality machine learning models.

An ad tech company is using an XGBoost model to classify its clickstream data. The company’s Data Scientist is asked to explain how the model works to a group of non-technical colleagues. What is a simple explanation the Data Scientist can provide?

Machine learning is a form of artificial intelligence that allows computers to learn from data, without being explicitly programmed. machine learning is a powerful tool for solving complex problems, and XGBoost is a popular machine learning algorithm. machine learning algorithms like XGBoost work by building a model based on training data, and then using that model to make predictions on new data. In the case of the ad tech company, the Data Scientist has used XGBoost to build a model that can classify clickstream data. This means that the model can look at new data and predict which category it belongs to. For example, the model might be able to predict whether a user is likely to click on an ad or not. The Data Scientist can explain how the model works by showing how it makes predictions on new data.

Machine learning is a method of teaching computers to learn from data, without being explicitly programmed. machine learning is a subset of artificial intelligence (AI). The XGBoost algorithm is a machine learning technique used to create models that predict outcomes by learning from past data. XGBoost is an implementation of gradient boosting, which is a machine learning technique for creating models that make predictions by combining the predictions of multiple individual models. The XGBoost algorithm is highly effective and is used by many organizations, including ad tech companies, to classify their data. The Data Scientist can explain how the XGBoost model works by providing a simple explanation of machine learning and how the XGBoost algorithm works. machine learning is a method of teaching computers to learn from data, without being explicitly programmed. 

 

An ML Engineer at a real estate startup wants to use a new quantitative feature for an existing ML model that predicts housing prices. Before adding the feature to the cleaned dataset, the Engineer wants to visualize the feature in order to check for outliers and overall distribution and skewness of the feature. What visualization technique should the ML Engineer use? 

The machine learning engineer at the real estate startup should use a visualization technique in order to check for outliers and overall distribution and skewness of the new quantitative feature. There are many different visualization techniques that could be used for this purpose, but two of the most effective are histograms and scatterplots. A histogram can show the distribution of values for the new feature, while a scatterplot can help to identify any outliers. By visualizing the data, the engineer will be able to ensure that the new feature is of high quality and will not impact the performance of the machine learning model.

When updating your weights using the loss function, what dictates how much change the weights should have?

In machine learning and data science, the learning rate is a parameter that dictates how much change the weights should have when updating them using the loss function. The learning rate is typically a small value between 0 and 1. A higher learning rate means that the weights are updated more quickly, which can lead to faster convergence but can also lead to instability. A lower learning rate means that the weights are updated more slowly, which can lead to slower convergence but can also help avoid overfitting. The optimal learning rate for a given problem can be found through trial and error. The bias term is another parameter that can affect the weight updates. The bias term is used to prevent overfitting by penalizing models that make too many assumptions about the data. The initial weights are also important, as they determine where the model starts on the optimization landscape. The batch size is another important parameter, as it defines how many training examples are used in each iteration of weight updates. A larger batch size can lead to faster convergence, but a smaller batch size can help avoid overfitting. Finding the optimal values for all of these parameters can be a challenge, but doing so is essential for training high-quality machine learning models.

The loss function is a key component of machine learning algorithms, as it determines how well the model is performing. When updating the weights using the loss function, the learning rate dictates how much change the weights should have. The learning rate is a hyperparameter that can be tuned to find the optimal value for the model. The bias term is another important factor that can influence the weights. The initial weights can also play a role in how much change the weights should have. The batch size is another important factor to consider when updating the weights using the loss function.

A data scientist wants to clean and merge two small datasets stored in CSV format. What tool can they use to merge these datasets together?

As a data scientist, you often need to work with multiple datasets in order to glean insights that would be hidden in any one dataset on its own. In order to do this, you need to be able to clean and merge datasets quickly and efficiently. One tool that can help you with this task is Pandas. Pandas is a Python library that is specifically designed for data analysis. It offers a wide range of features that make it well-suited for merging datasets, including the ability to read in CSV format, clean data, and merge datasets with ease. In addition, Pandas integrates well with other machine learning libraries such as Scikit-learn, making it a valuable tool for data scientists.

As a data scientist, one of the most important skills is knowing how to clean and merge datasets. This can be a tedious and time-consuming process, but it is essential for machine learning and data science projects. There are several tools that data scientists can use to merge datasets, but one of the most popular options is pandas. Pandas is a Python library that offers a wide range of functions for data manipulation and analysis. Additionally, pandas has built-in support for reading and writing CSV files. This makes it an ideal tool for merging small datasets stored in CSV format. With pandas, data scientists can quickly and easily clean and merge their data, giving them more time to focus on other aspects of their projects.

A real estate company is building a linear regression model to predict housing prices for different cities in the US. Which of the following is NOT a good metric to measure performance of their regression model?

Machine learning is a subset of data science that deals with the design and development of algorithms that can learn from and make predictions on data. Linear regression is a machine learning algorithm used to predict numerical values based on a linear relationship between input variables. When building a linear regression model, it is important to choose an appropriate metric to measure the performance of the model. The F1 score, R-squared value, and mean-squared error are all valid metrics for measuring the performance of a linear regression model. However, the mean absolute error is not a good metric to use for this purpose, as it does not take into account the direction of the prediction error (i.e., whether the predicted value is higher or lower than the actual value). As such, using the mean absolute error as a metric for evaluating the performance of a linear regression model could lead to inaccurate results.

A real estate company wants to provide its customers with a more accurate prediction of the final sale price for houses they are considering in various cities. To do this, the company wants to use a fully connected neural network trained on data from the previous ten years of home sales, as well as other features. What kind of machine learning problem does this situation most likely represent?

Answer: Regression

Which feature of Amazon SageMaker can you use for preprocessing the data?

 
 

Answer: Amazon Sagemaker Notebook instances

Amazon SageMaker enables developers and data scientists to build, train, tune, and deploy machine learning (ML) models at scale. You can deploy trained ML models for real-time or batch predictions on unseen data, a process known as inference. However, in most cases, the raw input data must be preprocessed and can’t be used directly for making predictions. This is because most ML models expect the data in a predefined format, so the raw data needs to be first cleaned and formatted in order for the ML model to process the data.  You can use the Amazon SageMaker built-in Scikit-learn library for preprocessing input data and then use the Amazon SageMaker built-in Linear Learner algorithm for predictions.

What setting, when creating an Amazon SageMaker notebook instance, can you use to install libraries and import data?

Answer: LifeCycle Configuration

You work for the largest coffee chain in the world. You’ve recently decided to source beans from a new market to create new blends and flavors. These beans come from 30 different growers, in 3 different countries. In order to keep a consistent flavor, you have each grower send samples of their beans to your tasting baristas who rate the beans on 20 different dimensions. You now need to group the beans together so the supply can be diversified yet the flavor of the final product kept as consistent as possible.
What is one way you could convert this business situation into a machine learning problem?

 
 

Answer:

In which phase of the ML pipeline does the machine learn from the data?

 
 
 

Answer: Model Training

A text analytics company is developing a text classification model to detect whether a document involves offensive content or not. The training dataset included ten non-offensive documents for every one offensive document. Their model resulted in an accuracy score of 94%.
What can we conclude from this result?

Answer: Accuracy is the wrong metric here, because it can be heavily influenced by the large class (non-offensive documents).

A Machine Learning Engineer is creating a regression model for forecasting company revenue based on an internal dataset made up of past sales and other related data.

 

What metric should the Engineer use to evaluate the ML model?

 
Answer: Root Mean Squared error (RMSE)
Root Mean Square Error (RMSE) is the standard deviation of the residuals (prediction errors). Residuals are a measure of how far from the regression line data points are; RMSE is a measure of how spread out these residuals are. In other words, it tells you how concentrated the data is around the line of best fit.

An ML scientist has built a decision tree model using scikit-learn with 1,000 trees. The training accuracy for the model was 99.2% and the test accuracy was 70.3%. Should the Scientist use this model in production?

 
Answer:  No, because it is not generalizing well on the test set
 
 
 

The curse of dimensionality relates to which of the following?

 
Answer: A – A high number of features in a dataset

 

The curse of dimensionality relates to a high number of features in a dataset.

Curse of Dimensionality describes the explosive nature of increasing data dimensions and its resulting exponential increase in computational efforts required for its processing and/or analysis. This term was first introduced by Richard E.

A Data Scientist wants to include “month” as a categorical column in a training dataset for an ML model that is being built. However, the ML algorithm gives an error when the column is added to the training data. What should the Data Scientist do to add this column?

 

Answer:

StandardScaler standardizes a feature by subtracting the mean and then scaling to unit variance. Unit variance means dividing all the values by the standard deviation. StandardScaler does not meet the strict definition of scale I introduced earlier.

What is the primary reason that one might want to pick either random search or Bayesian optimization over grid search when performing hyperparameter optimization?

 
 
Answer: Random search and Bayesian methods leave smaller unexplored regions than grid searches
 
 
 

A Data Scientist trained an XGBoost model to classify internal documents for further inquiry, and now wants to evaluate the model’s performance by looking at the results visually. What technique should the Data Scientist use in this situation?

 
 
 

Machine Learning Pipeline Goals

Machine Learning Pipeline Goals
Machine Learning Pipeline Goals

Common Machine Learning Use Cases

Common Machine LEarning USe Cases
Common Machine LEarning USe Cases

What is a machine learning model?

What is a machine learning model?
What is a machine learning model?

What are features and weights meaning in Machine Learning?

Features meaning in ML
Features meaning in ML

 

Weights in ML
Weights in ML

 

Machine Learning Features and Weights
Machine Learning Features and Weights

Reference: Wiki

PRE-PROCESSING AND FEATURE ENGINEERING

Visualization

image

image

Outliers

Missing Value

Feature Engineering

 

Source: https://github.com/mortezakiadi/ML-Pipeline/wiki

How to Choose the right Sagemaker built-in algorithm?

How to chose the right built in algorithm in SageMaker?
How to chose the right built in algorithm in SageMaker?
Guide to choosing the right unsupervised learning algorithm
Guide to choosing the right unsupervised learning algorithm

 

Choosing the right  ML algorithm based on Data Type
Choosing the right ML algorithm based on Data Type

 

Choosing the right ML algo based on data type
Choosing the right ML algo based on data type

This is a general guide for choosing which algorithm to use depending on what business problem you have and what data you have. 

Machine Learning Deployment and Monitoring

Deployment and Monitoring

Machine Learning Model Performance Evaluation Metrics

image

Machine Learning Breaking News and Top Stories

  • [P] FOSS MLOps tool that simplifies handing off models between ML engineers and app developers
    by /u/iamjessew (Machine Learning) on March 18, 2024 at 2:29 pm

    Hi all - We just open sourced our tool for handing models off between our ml engineers/data scientists and our app dev team. It solves a ton of problems that we experienced, specifically having to maintain two different pipelines and having to constantly hunt down model assets. Curious to know what you think, you can check it out here: https://kitops.ml submitted by /u/iamjessew [link] [comments]

  • [D] What development methodology does your place follow?
    by /u/notEVOLVED (Machine Learning) on March 18, 2024 at 2:26 pm

    Do they follow Agile? Does it work well with ML based development? Particularly, how do you deal with the uncertainty in producing the deliverables for an ML project? (uncertainty in reaching the desired performance within the set time) submitted by /u/notEVOLVED [link] [comments]

  • [D] Deep Regression problem on facial landmarks
    by /u/TERMINERO (Machine Learning) on March 18, 2024 at 1:39 pm

    Hello to all, I'm currently in the process of researching and trying to implement a model that takes in input the facial landmarkers of mediapipe (represented as a point cloud) and what i'm trying to obtain are values for the slider of the open-source tool makehuman (http://www.makehumancommunity.org/) such as nose width and ect... I've built a dataset of 10000 avatar faces from which i've taken the facial landmarks and the value of sliders used for creating them as labels. I'm already looking at something like PointNet++ as implemented in torch_geometric tutorials (https://pytorch-geometric.readthedocs.io/en/latest/tutorial/point_cloud.html) by modifing the last layer, the loss function and the metrics in order to obtain a regressor. I've also tried to transform the task in a classification task by transforming the sliders values in 20 categories of value from -1 to 1 by 0.1 in 0.1 but both implementation return sub-optimal results with mean accuracy for the classification task of 0.0015 and mean mse of 0.06 for the regression task. Any advise on architecture to use or another way to interpret the input data are welcome! submitted by /u/TERMINERO [link] [comments]

  • [Discussion] Data formatting from PDF to QnA using LLM
    by /u/Medium_Alternative50 (Machine Learning) on March 18, 2024 at 1:34 pm

    Data preparation is hard, this is something that I observed when I wanted to fine-tune a model based on data present in a bunch of pdfs, I needed to convert the content in the pdf to a QnA dataset, but obviously I won't do it manually. I came across this YouTube video (youtube.com/watch?v=fYyZiRi6yNE) this person has created a script here: https://github.com/Aemon-Algiz/DatesetExtraction/blob/main/BookParse.py This script converts the pdfs into text and then uses an LLM to make a question and answer pair from that text. Does everyone use this technique to structure the data, or is there a more efficient way to do this? Also, as I'm learning and getting more into LLMs I'm able to observe the challenges, but I would also like to know what other challenges are on the way and faced mostly. submitted by /u/Medium_Alternative50 [link] [comments]

  • [D] A100
    by /u/Ettaross (Machine Learning) on March 18, 2024 at 1:17 pm

    Hi, I have an Nvidia A100 80GB and I want to purchase the remaining components for it. I use it for LLM and stable diffusion. I'm considering this: Motherboard: Asrock ROMED4ID-2T RAM: 128 GB CPU: I don't know which processor to buy that wouldn't be very expensive. The whole thing will be in a normal case because I have built cooling for the A100. What do you think about it? Thank you very much! submitted by /u/Ettaross [link] [comments]

  • [D] An idea about critic network for neural network parameters
    by /u/NoteDance (Machine Learning) on March 18, 2024 at 1:08 pm

    Training the parameter critic network: This method is used to train the critic network for neural network parameters, such as the parameters of the last layer of the neural network. The neural network is considered as the environment and actor, where the state is the parameters of the neural network, the action is to update the parameters, and the reward is -bl, where bl is the batch loss. The input to the critic network is the parameters of the neural network, and the output is a scalar. The TD error is the loss function of the critic network, where TD = -bl + critic(param’) - critic(param), with no discount needed. The critic(param) = -blt - blt+1 … - bln, where t is the number of times the neural network parameters have been updated, and n is the total number of batches, so the later the state(param), the greater its state value. Using the parameter critic network: When we train the neural network with new methods or data, we can use the parameter critic network to output state value of these neural network parameters. The new loss is loss + αcritic(param), where we aim to minimize loss and critic(param) in the new loss. Here, α is an adjustable hyperparameter, and this new loss will measure both loss and critic(param), ensuring that the neural network maintains good performance on old data. submitted by /u/NoteDance [link] [comments]

  • [D] ICML 2024 Rebuttals Thread
    by /u/South-Conference-395 (Machine Learning) on March 18, 2024 at 12:49 pm

    Hello everyone, As we are close to the release of the reviews, I am creating this thread to gather thoughts/ questions/ suggestions of fellow researchers. Good luck everyone! submitted by /u/South-Conference-395 [link] [comments]

  • [D] Question about OLLAMA and Hardware Configuration for LLM Inference
    by /u/King_of_Junk (Machine Learning) on March 18, 2024 at 12:17 pm

    I'm planning to use OLLAMA locally with various LLMs, including LLaMA 70B with personal embedded parameters and data. I'd like to avoid the cost of a pricey A100 80GB or higher card and I'm considering using only 4 x 3090 for inference. Would that configuration work fine? Also, I'd like to know if OLLAMA automatically manages memory across the cards when using multiple GPUs. submitted by /u/King_of_Junk [link] [comments]

  • [D] papers that only evaluate on cifar10
    by /u/audiencevote (Machine Learning) on March 18, 2024 at 12:08 pm

    Hi everyone! While reviewing for NeurIPS 2024, one of the things I keep noticing is that a lot of papers only evaluate on datasets of very small size, like Cifar-10. his feels weird to me: I consider Cifar10 to be a toy-dataset and testbed for my methods, not something I'd use to show that my method actually works/is relevant in practice. So my first intuition is always "this approach probably does not scale to larger datasets". I mean, ImageNet is 12 years old now, and I've personally been giving results on imagenet for my papers since ~8 years. most computer vision applications I know require larger resolutions than just 32x32. it's also my impression that almost all of the "good" papers I read have results on larger scale data. But given how often I encounter this situation, I have to wonder: am I just working in a very privileged environment, or are the manuscript-authors just lazy? How much faith do you have in papers that only evaluate on MNIST and CIFAR10? submitted by /u/audiencevote [link] [comments]

  • [D] Theoretical Paper about Transformers
    by /u/Massive_Horror9038 (Machine Learning) on March 18, 2024 at 11:29 am

    I'm going to start a study group focused on large language models. The participants are PhD students in Computer Science with a math background. I would like to first study some theoretical properties of transformers (or attention). Maybe some of the students still do not know exactly how a transformer is formulated, so I'll also need to discuss that. Do you have any suggestions of papers with an theoretical analysis of transformers (attention)? The most popular paper for attention is "Attention is all you need", but they only present the architecture and run experiments. submitted by /u/Massive_Horror9038 [link] [comments]

  • In 2024 which library is best for time series forecasting and anomaly detection? [D]
    by /u/ThakkidiMundan (Machine Learning) on March 18, 2024 at 11:03 am

    I am working on a project where I am dealing with identifying anomalies in a time series data. I came across Facebook prophet, but sadly it is no longer maintained since 2023. They suggested neuralprophet, nixtla as alternative. When looking for alternatives I came across Kats from Facebook which has prophet support built-in. Which tool/lib experienced members here would recommend, for building a production level system for anomaly detection on high volume of timeseries data? submitted by /u/ThakkidiMundan [link] [comments]

  • [R] Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence
    by /u/SunsetOneSix (Machine Learning) on March 18, 2024 at 10:51 am

    Paper: https://arxiv.org/abs/2403.05996 Abstract: We show that deep reinforcement learning can maintain its ability to learn without resetting network parameters in settings where the number of gradient updates greatly exceeds the number of environment samples. Under such large update-to-data ratios, a recent study by Nikishin et al. (2022) suggested the emergence of a primacy bias, in which agents overfit early interactions and downplay later experience, impairing their ability to learn. In this work, we dissect the phenomena underlying the primacy bias. We inspect the early stages of training that ought to cause the failure to learn and find that a fundamental challenge is a long-standing acquaintance: value overestimation. Overinflated Q-values are found not only on out-of-distribution but also in-distribution data and can be traced to unseen action prediction propelled by optimizer momentum. We employ a simple unit-ball normalization that enables learning under large update ratios, show its efficacy on the widely used dm_control suite, and obtain strong performance on the challenging dog tasks, competitive with model-based approaches. Our results question, in parts, the prior explanation for sub-optimal learning due to overfitting on early data. submitted by /u/SunsetOneSix [link] [comments]

  • [R] Arrows of Time for Large Language Models
    by /u/SunsetOneSix (Machine Learning) on March 18, 2024 at 10:42 am

    Paper: https://arxiv.org/abs/2401.17505 Abstract: We study the probabilistic modeling performed by Autoregressive Large Language Models through the angle of time directionality. We empirically find a time asymmetry exhibited by such models in their ability to model natural language: a difference in the average log-perplexity when trying to predict the next token versus when trying to predict the previous one. This difference is at the same time subtle and very consistent across various modalities (language, model size, training time, ...). Theoretically, this is surprising: from an information-theoretic point of view, there should be no such difference. We provide a theoretical framework to explain how such an asymmetry can appear from sparsity and computational complexity considerations, and outline a number of perspectives opened by our results. submitted by /u/SunsetOneSix [link] [comments]

  • [D] Is there anyone using Neural corpus indexer(NCI)?
    by /u/Capital_Reply_7838 (Machine Learning) on March 18, 2024 at 10:24 am

    I thought NCI, nominated as one of outstanding papers, would change whole things about retrieval tasks. However, It seems not so far(for me). Does it have some critical problems? submitted by /u/Capital_Reply_7838 [link] [comments]

  • [D] When your use of AI for summary didn't come out right. A published Elsevier research paper
    by /u/vvkuka (Machine Learning) on March 18, 2024 at 10:14 am

    submitted by /u/vvkuka [link] [comments]

  • [P] Hugging Face Demo App: Explainable Generative AI for Tabular Data
    by /u/MLC_Money (Machine Learning) on March 18, 2024 at 7:51 am

    Hello folks, I invite you to try the hugging face demo app of our work on explainable and generative AI for tabular data. https://huggingface.co/spaces/CaglarAytekin/LEURN Briefly, the app takes a training table including features and target, lets you select what to predict and select categorical columns. Then you can train the neural network with selected hyperparameters. Once the training is finished, you can upload a test data (excel or csv with only features, no target), and select a row to run the neural network on and explain the decisions. You can also seamlessly generate new data based on what your neural network learned from the training. Clear usage instructions can be found from the above app. Related research paper: https://arxiv.org/abs/2303.14937 Open-source code: https://github.com/CaglarAytekin/LEURN/ contact: caglar@deepcause.ai ​ submitted by /u/MLC_Money [link] [comments]

  • [D] Deploying Mistral 7B - Quantization Methods, Hosting Options etc. (for the GPU poor)
    by /u/Aggravating-Floor-38 (Machine Learning) on March 18, 2024 at 7:19 am

    I'm trying to deploy a Mistral 7B api endpoint for a RAG application I'm building. A few major things I'm confused about - I'm GPU poor 🙁 so was planning on using AWS sagemaker to deploy the model - the 2 month free plan has 125 hours of m4.xlarge or m5.xlarge instance per month on Inference - would that be enough to set up an endpoint for quantized mistral (I'm thinking 5-bit)? And like if you don't have a GPU yourself and are a broke student what's the go-to for model hosting? Just need it for my final year project so it's not a long-term thing. Also I'm confused about the quantization types and methods - first out of AWQ, GGUF and GPTQ I'm leaning towards AWQ for it's efficiency, but I've heard that the downside to it is the limited support? If I just want to deploy TheBloke's AWQ model, that shouldn't be anything particularly difficult right? Also is that the only downside to AWQ of could it potentially require more RAM? Also was looking at the deployment methods themselves - openllama, vllama etc. I was leaning towards vllama because again efficiency - has like double the throughput of Huggingface TGI, which I was using until this point. But then saw someone say something about how it might require more VRAM? So advice please - What are my hosting options, what quantization method should I be using, within these v limited resources what are my options for making this model as fast as possible? Note: Ik for most cases it can't be helped and I will have to compromise on model speed, but I want to use the model for metadata extraction of chunks too, and if I'm feeding this model 100s of chunks, I need it to not be slow to the point of impracticality lol. submitted by /u/Aggravating-Floor-38 [link] [comments]

  • [D] Math behind neural operator for PDE solving
    by /u/zarzor_2010 (Machine Learning) on March 18, 2024 at 5:28 am

    It seems that almost all Neural operator papers, like FNO and others, always use the layers in the form of vt+1(x) = σ ( W vt(x)+ ∫ K(x, y) vt(y) dy ) , where t is the layer. Any proof why this works? Looks like many explanations that the form is similar to Green's functions solution, but then each method uses a different Kernel without concrete relation to Green's functions or why it works. submitted by /u/zarzor_2010 [link] [comments]

  • [Discussion] ICML 24
    by /u/Objective_Whole_1406 (Machine Learning) on March 18, 2024 at 4:44 am

    We had a post to discuss AAAI. But, we don't seem to have one for ICML. So, this post is for ICML discussion and other stuff. The reviews are going to be put in 2 days. Cheers! submitted by /u/Objective_Whole_1406 [link] [comments]

  • [D] xAI’s Qdrant…Why?
    by /u/titani0us (Machine Learning) on March 18, 2024 at 4:11 am

    Maybe I just haven't put the time in to understanding it, but I'm struggling to understand how Qdrant is any better than OpenSearch/ElasticSearch? OS/ES both use HNSW, and they both use the same KNN oss implementation that is extremely performant. What does Qdrant have "out of the box" these existing — and widely adopted — options don't have? submitted by /u/titani0us [link] [comments]




Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses
Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

How do you export data from BigQuery to a CSV file?

Google Cloud’s BigQuery is a powerful tool for storing and querying large data sets. However, sometimes you may need to export data from BigQuery in order to perform additional analysis or simply to have a backup. Thankfully, Google Cloud makes it easy to export data from BigQuery to a CSV file.

  • The first step is to select the dataset that you want to export.
  • Next, click on the “Export Table” button. In the pop-up window, select “CSV” as the file format and choose a location to save the file.
  • Finally, click on the “Export” button and Google Cloud will begin exporting the data.
  • Depending on the size of the data set, this may take several minutes. Once the export is complete, you will have a CSV file containing all of the data from BigQuery.

Alternatively, Simply run the following command:

“bq extract –destination_format=CSV [dataset_name] [table_name]”.

This will export your data to a CSV file in Google Cloud Storage. You can then download the file from Google Cloud Storage and use it in another program. Alternatively, you can use the “bq load” command to load your data directly into another Google Cloud service, such as Google Sheets.

Person climbing a staircase. Learn Data Science from Scratch: online program with 21 courses

What is the Difference Between Mini-Batch and Full-Batch in Machine Learning?

In the field of machine learning, there are two types of batch sizes that are commonly used: mini-batch and full-batch. Both have their pros and cons, and the choice of which to use depends on the situation. Here’s a quick rundown of the differences between mini-batch and full-batch in machine learning.

Mini-Batch Machine Learning
Mini-batch machine learning is a type of batch processing where the data is divided into small batches before being fed into the machine learning algorithm. The advantage of mini-batch machine learning is that it can provide more accurate results than full-batch machine learning, since the data is less likely to be affected by outliers. However, the disadvantage of mini-batch machine learning is that it can be slower than full-batch machine learning, since each batch has to be processed separately.

Full-Batch Machine Learning
Full-batch machine learning is a type of batch processing where the entire dataset is fed into the machine learning algorithm at once. The advantage of full-batch machine learning is that it is faster than mini-batch machine learning, since all the data can be processed simultaneously. However, the disadvantage of full-batch machine learning is that it can be less accurate than mini-batch machine learning, since outliers in the dataset can have a greater impact on the results.

So, which should you use? It depends on your needs. If accuracy is more important than speed, then mini-batch machine learning is the way to go. On the other hand, if speed is more important than accuracy, then full-batch machine learning is the way to go.

The Difference Between Mini-Batch and Full-Batch Learning

In machine learning, there are two main types of batch learning: mini-batch and full-batch. Both types of batch learning algorithms have their own pros and cons that data scientists should be aware of. In this blog post, we’ll take a look at the difference between mini-batch and full-batch learning so you can make an informed decision about which type of algorithm is right for your project.

Mini-batch learning is a type of batch learning that operates on small subsets of the training data, typically referred to as mini-batches. The advantage of mini-batch learning is that it can be parallelized across multiple processors or devices, which makes training much faster than full-batch training. Another advantage is that mini-batches can be generated on the fly from a larger dataset, which is especially helpful if the entire dataset doesn’t fit into memory. However, one downside of mini-batch learning is that it can sometimes lead to suboptimal results due to its stochastic nature.

Full-Batch Learning
Full-batch learning is a type of batch learning that operates on the entire training dataset at once. The advantage of full-batch learning is that it converges to the global optimum more quickly than mini-batch or stochastic gradient descent (SGD) methods. However, the disadvantage of full-batch learning is that it is very slow and doesn’t scale well to large datasets. Additionally, full-batch methods can’t be parallelized across multiple processors or devices due to their sequential nature.

So, which type of batch learning algorithm is right for your project? If you’re working with a small dataset, then full-batch learning might be your best bet. However, if you’re working with a large dataset or need to train your model quickly, then mini=batch or SGD might be better suited for your needs. As always, it’s important to experiment with different algorithms and tuning parameters to see what works best for your particular problem.

2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams

Welcome to AWS Certification Machine Learning Specialty (MLS-C01) Practice Exams! 

This book is designed to help you prepare for the AWS Certified Machine Learning – Specialty (MLS-C01) exam and earn your AWS certification. The AWS Certified Machine Learning – Specialty (MLS-C01) exam is designed for individuals who have a strong understanding of machine learning concepts and techniques, and who can design, build, and deploy machine learning models on the AWS platform.

In this book, you will find a series of practice exams that are designed to mimic the format and content of the actual MLS-C01 exam. Each practice exam includes a set of multiple choice and multiple response questions that cover a range of topics, including machine learning concepts, techniques, and algorithms, as well as the AWS services and tools used to build and deploy machine learning models.

By working through these practice exams, you can test your knowledge, identify areas where you need further study, and gain confidence in your ability to pass the MLS-C01 exam. Whether you are a machine learning professional looking to earn your AWS certification or a student preparing for a career in machine learning, this book is an essential resource for your exam preparation.

2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams
2023 AWS Certified Machine Learning Specialty (MLS-C01) Practice Exams

What is the best Japanese natural language processing (NLP) library?

NLP is a field of computer science and artificial intelligence that deals with the interactions between computers and human languages. NLP algorithms are used to process and analyze large amounts of natural language data. Japanese NLP libraries are used to develop applications that can understand and respond to Japanese text.

The best Japanese NLP library depends on your application’s needs.

For example, if you are developing a machine translation application, you will need a library that supports word sense disambiguation and part-of-speech tagging. If you are developing a chatbot, you will need a library that supports sentence analysis and dialogue management. In general, Japanese NLP libraries can be divided into three categories: rule-based systems, statistical systems, and hybrid systems.

Rule-based systems rely on linguistic rules to process language data.

Statistical systems use statistical models to process language data.

Hybrid systems use a combination of linguistic rules and statistical models to process language data.

The best Japanese NLP library for your application will depend on the type of NLP tasks you need to perform and your resources (e.g., time, data, computing power).

XGBoost is a powerful tool that has a wide range of applications in the real world. XGBoost is a machine learning algorithm that is used to improve the performance of other machine learning algorithms.

XGBoost has been used to improve the performance of data science models in a variety of fields, including healthcare, finance, and retail.

In healthcare, XGBoost has been used to predict patient outcomes, such as length of stay in the hospital and mortality rates.

In finance, XGBoost has been used to predict stock prices and credit card fraud.

In retail, XGBoost has been used to improve customer segmentation and product recommendations.

XGBoost is a versatile tool that can be used to improve the performance of machine learning models in many different fields.

Machine Learning For Dummies  on iOs:  https://apps.apple.com/us/app/machinelearning-for-dummies-p/id1610947211

Machine Learning For Dummies on Windowshttps://www.microsoft.com/en-ca/p/machinelearning-for-dummies-ml-ai-ops-on-aws-azure-gcp/9p6f030tb0mt?

Machine Learning For Dummies Web/Android on Amazon: https://www.amazon.com/gp/product/B09TZ4H8V6

#MachineLearning #AI #ArtificialIntelligence #ML #MachineLearningForDummies #MLOPS #NLP #ComputerVision #AWSMachineLEarning #AzureAI #GCPML

Can AI predicts the tournament winner and golden booth for the FIFA World Cup Football Soccer 2022 in Qatar?

What are some ways we can use machine learning and artificial intelligence for algorithmic trading in the stock market?

How do we know that the Top 3 Voice Recognition Devices like Siri Alexa and Ok Google are not spying on us?

What are some good datasets for Data Science and Machine Learning?

What are popular hobbies among Software Engineers?

What is the programming model and best language for Hadoop and Spark? Python or Java?

Google PageRank

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Hadoop is an open-source software framework for storing data and running applications on clusters of commodity hardware. It provides massive storage for any kind of data, enormous processing power and the ability to handle virtually limitless concurrent tasks or jobs. Apache Hadoop is used mainly for Data Analysis

Apache Spark is an open-source distributed general-purpose cluster-computing framework. Spark provides an interface for programming entire clusters with implicit data parallelism and fault tolerance

The question is Which programming language is good to drive Hadoop and Spark?

The programming model for developing hadoop based applications is the map reduce. In other words, MapReduce is the processing layer of Hadoop.
MapReduce programming model is designed for processing large volumes of data in parallel by dividing the work into a set of independent tasks. Hadoop MapReduce is a software framework for easily writing an application that processes the vast amount of structured and unstructured data stored in the Hadoop Distributed FileSystem (HDFS). The biggest advantage of map reduce is to make data processing on multiple computing nodes easy. Under the Map reduce model, data processing primitives are called Mapper and Reducers.

Spark is written in Scala and Hadoop is written in Java.


The key difference between Hadoop MapReduce and Spark lies in the approach to processing: Spark can do it in-memory, while Hadoop MapReduce has to read from and write to a disk. As a result, the speed of processing differs significantly – Spark may be up to 100 times faster.

In-memory processing is faster when compared to Hadoop, as there is no time spent in moving data/processes in and out of the disk. Spark is 100 times faster than MapReduce as everything is done here in memory.

Spark’s hardware is more expensive than Hadoop MapReduce because it’s hardware needs a lot of RAM.


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

Hadoop runs on Linux, it means that you must have knowldge of linux.

Java is important for hadoop because:

  • There are some advanced features that are only available via the Java API.
  • The ability to go deep into the Hadoop coding and figure out what’s going wrong.

In both these situations, Java becomes very important.
As a developer, you can enjoy many advanced features of Spark and Hadoop if you start with their native languages (Java and Scala).

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

What Python Offers for Hadoop and Spark?

  • Simple syntax– Python offers simple syntax which shows it is more user friendly than other two languages.
  • Easy to learn – Python syntax are like English languages. So, it much more easier to learn it and master it.
  • Large community support – Unlike Scala, Python has huge community (active), which we will help you to solve your queries.
  • Offers Libraries, frameworks and packages – Python has huge number of Scientific packages, libraries and framework, which are helping you to work in any environment of Hadoop and Spark.
  • Python Compatibility with Hadoop – A package called PyDoop offers access to the HDFS API for Hadoop and hence it allows to write Hadoop MapReduce program and application.

  • Hadoop is based off of Java (then so e.g. non-Hadoop yet still a Big-Data technology like the ElasticSearch engine, too – even though it processes JSON REST requests)
  • Spark is created off of Scala although pySpark (the lovechild of Python and Spark technologies of course) has gained a lot of momentum as of late.


If you are planning for Hadoop Data Analyst, Python is preferable given that it has many libraries to perform advanced analytics and also you can use Spark to perform advanced analytics and implement machine learning techniques using pyspark API.


The key Value pair is the record entity that MapReduce job receives for execution. In MapReduce process, before passing the data to the mapper, data should be first converted into key-value pairs as mapper only understands key-value pairs of data.
key-value pairs in Hadoop MapReduce is generated as follows:

Resources:

1- Quora

2- Wikipedia

3- Data Flair

AWS Developer and Deployment Theory: Facts and Summaries and Questions/Answers

AWS Certification Exam Preparation

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

AWS Developer and Deployment Theory: Facts and Summaries and Questions/Answers

AWS Developer – Deployment Theory Facts and summaries, Top 80 AWS Developer  DVA-C02 Theory Questions and Answers Dump

Definition 1: The AWS Developer is responsible for designing, deploying, and developing cloud applications on AWS platform

Definition 2: The AWS Developer Tools is a set of services designed to enable developers and IT operations professionals practicing DevOps to rapidly and safely deliver software.

The AWS Certified Developer Associate certification is a widely recognized certification that validates a candidate’s expertise in developing and maintaining applications on the Amazon Web Services (AWS) platform.

The certification is about to undergo a major change with the introduction of the new exam version DVA-C02, replacing the current DVA-C01. In this article, we will discuss the differences between the two exams and what candidates should consider in terms of preparation for the new DVA-C02 exam.

Quick facts


  • What’s happening?
  • The DVA-C01 exam is being replaced by the DVA-C02 exam.
  • When is this taking place?
  • The last day to take the current exam is February 27th, 2023 and the first day to take the new exam is February 28th, 2023.
  • What’s the difference?
  • The new exam features some new AWS services and features.

Main differences between DVA-C01 and DVA-C02

The table below details the differences between the DVA-C01 and DVA-C02 exams domains and weightings:

In terms of the exam content weightings, the DVA-C02 exam places a greater emphasis on deployment and management, with a slightly reduced emphasis on development and refactoring. This shift reflects the increased importance of operations and management in cloud computing, as well as the need for developers to have a strong understanding of how to deploy and maintain applications on the AWS platform.


AI Unraveled: Demystifying Frequently Asked Questions on Artificial Intelligence (OpenAI, ChatGPT, Google Bard, Generative AI, Discriminative AI, xAI, LLMs, GPUs, Machine Learning, NLP, Promp Engineering)

One major difference between the two exams is the focus on the latest AWS services and features. The DVA-C02 exam covers around 57 services vs only 33 services in the DVA-C01. This reflects the rapidly evolving AWS ecosystem and the need for developers to be up-to-date with the latest services and features in order to effectively build and maintain applications on the platform.

Click the image above to watch our video about the NEW AWS Developer Associate Exam DVA-C02 from our youtube channel

Training resources for the AWS Developer Associate

If you are looking for an all-in-one solution to help you prepare for the AWS Cloud Practitioner Certification Exam, look no further than this AWS Cloud Practitioner CCP CLF-C02 book

In terms of preparation for the DVA-C02 exam, we strongly recommend enrolling in our on-demand training courses for the AWS Developer Associate certification. It is important for candidates to familiarize themselves with the latest AWS services and features, as well as the updated exam content weightings. Practical experience working with AWS services and hands-on experimentation with new services and features will be key to success on the exam. Candidates should also focus on their understanding of security best practices, access control, and compliance, as these topics will carry a greater weight in the new exam.

Frequently asked questions – FAQs:

In conclusion, the change from the DVA-C01 to the DVA-C02 exam represents a major shift in the focus and content of the AWS Certified Developer Associate certification. Candidates preparing for the new exam should focus on familiarizing themselves with the latest AWS services and features, as well as the updated exam content weightings, and placing a strong emphasis on security, governance, and compliance.

With the right preparation and focus, candidates can successfully navigate the changes in the DVA-C02 exam and maintain their status as a certified AWS Developer Associate.

Download AWS Certified Developer Associate Mock Exam Pro App for:

AWS Developer and Deployment Theory:  Facts and Summaries and Questions/Answers
aws certified developer associate exam prep

All Platforms (PWA) –  Android –  iOSWindows 10 

AWS Developer and Deployment Theory Facts and summaries

AWS Developer and Deployment Theory:  Facts and Summaries and Questions/Answers
AWS Developer Associate DVA-C02 Exam Prep

    1. Continuous Integration is about integrating or merging the code changes frequently, at least once per day. It enables multiple devs to work on the same application.
    2. Continuous delivery is all about automating the build, test, and deployment functions.
    3. Continuous Deployment fully automates the entire release process, code is deployed into Production as soon as it has successfully passed through the release pipeline.
    4. AWS CodePipeline is a continuous integration/Continuous delivery service:
      • It automates your end-to-end software release process based on user defines workflow
      • It can be configured to automatically trigger your pipeline as soon as a change is detected in your source code repository
      • It integrates with other services from AWS like CodeBuild and CodeDeploy, as well as third party custom plug-ins.
    5. AWS CodeBuild is a fully managed build service. It can build source code, run tests and produce software packages based on commands that you define yourself.
    6. Dy default the buildspec.yml defines the build commands and settings used by CodeBuild to run your build.
    7. AWS CodeDeploy is a fully managed automated deployment service and can be used as part of a Continuous Delivery or Continuous Deployment process.
    8. There are 2 types of deployment approach:
      • In-place or Rolling update- you stop the application on each host and deploy the latest code. EC2 and on premise systems only. To roll back, you must re-deploy the previous version of the application.
      • Blue/Green : New instances are provisioned and the new application is deployed to these new instances. Traffic is routed to the new instances according to your own schedule. Supported for EC2, on-premise systems and Lambda functions. Rollback is easy, just route the traffic back to the original instances. Blue is active deployment, green is new release.
    9. Docker allows you to package your software into Containers which you can run in Elastic Container Service (ECS)
    10.  A docker Container includes everything the software needs to run including code, libraries, runtime and environment variables etc..
    11.  A special file called Dockerfile is used to specify the instructions needed to assemble your Docker image.
    12.  Once built, Docker images can be stored in Elastic Container Registry (ECR) and ECS can then use the image to launch Docker Containers.
    13. AWS CodeCommit is based on Git. It provides centralized repositories for all your code, binaries, images, and libraries.
    14. CodeCommit tracks and manages code changes. It maintains version history.
    15. CodeCommit manages updates from multiple sources and enables collaboration.
    16. To support CORS, API resource needs to implement an OPTIONS method that can respond to the OPTIONS preflight request with following headers:

      • Access-Control-Allow-Headers
      • Access-Control-Allow-Origin
      • Access-Control-Allow-Methods

    17. You have a legacy application that works via XML messages. You need to place the application behind the API gateway in order for customers to make API calls. Which of the following would you need to configure?
      You will need to work with the Request and Response Data mapping.
    18. Your application currently points to several Lambda functions in AWS. A change is being made to one of the Lambda functions. You need to ensure that application traffic is shifted slowly from one Lambda function to the other. Which of the following steps would you carry out?
      • Create an ALIAS with the –routing-config parameter
      • Update the ALIAS with the –routing-config parameter

      By default, an alias points to a single Lambda function version. When the alias is updated to point to a different function version, incoming request traffic in turn instantly points to the updated version. This exposes that alias to any potential instabilities introduced by the new version. To minimize this impact, you can implement the routing-config parameter of the Lambda alias that allows you to point to two different versions of the Lambda function and dictate what percentage of incoming traffic is sent to each version.

    19. AWS CodeDeploy: The AppSpec file defines all the parameters needed for the deployment e.g. location of application files and pre/post deployment validation tests to run.
    20. For Ec2 / On Premise systems, the appspec.yml file must be placed in the root directory of your revision (the same folder that contains your application code). Written in YAML.
    21. For Lambda and ECS deployment, the AppSpec file can be YAML or JSON
    22. Visual workflows are automatically created when working with which Step Functions
    23. API Gateway stages store configuration for deployment. An API Gateway Stage refers to A snapshot of your API
    24. AWS SWF Services SWF guarantees delivery order of messages/tasks
    25. Blue/Green Deployments with CodeDeploy on AWS Lambda can happen in multiple ways. Which of these is a potential option? Linear, All at once, Canary
    26. X-Ray Filter Expressions allow you to search through request information using characteristics like URL Paths, Trace ID, Annotations
    27. S3 has eventual consistency for overwrite PUTS and DELETES.
    28. What can you do to ensure the most recent version of your Lambda functions is in CodeDeploy?
      Specify the version to be deployed in AppSpec file.

      https://docs.aws.amazon.com/codedeploy/latest/userguide/application-specification-files.htmlAppSpec Files on an Amazon ECS Compute Platform

      If your application uses the Amazon ECS compute platform, the AppSpec file can be formatted with either YAML or JSON. It can also be typed directly into an editor in the console. The AppSpec file is used to specify:

      The name of the Amazon ECS service and the container name and port used to direct traffic to the new task set. The functions to be used as validation tests. You can run validation Lambda functions after deployment lifecycle events. For more information, see AppSpec ‘hooks’ Section for an Amazon ECS Deployment, AppSpec File Structure for Amazon ECS Deployments , and AppSpec File Example for an Amazon ECS Deployment .


    Top
    Reference: AWS Developer Tools




    AWS Developer and Deployment Theory: Top 80 Questions and Answers Dump

    Q0: Which AWS service can be used to compile source code, run tests and package code?

    • A. CodePipeline
    • B. CodeCommit
    • C. CodeBuild
    • D. CodeDeploy


    Answer: C.

    Reference: AWS CodeBuild


    Top

    Q1: How can your prevent CloudFormation from deleting your entire stack on failure? (Choose 2)

    • A. Set the Rollback on failure radio button to No in the CloudFormation console
    • B. Set Termination Protection to Enabled in the CloudFormation console
    • C. Use the –disable-rollback flag with the AWS CLI
    • D. Use the –enable-termination-protection protection flag with the AWS CLI


    Answer: A. and C.

    Reference: Protecting a Stack From Being Deleted

    Top

    Q2: Which of the following practices allows multiple developers working on the same application to merge code changes frequently, without impacting each other and enables the identification of bugs early on in the release process?

    • A. Continuous Integration
    • B. Continuous Deployment
    • C. Continuous Delivery
    • D. Continuous Development


    Answer: A

    Reference: What is Continuous Integration?

    Top

    Q3: When deploying application code to EC2, the AppSpec file can be written in which language?

    • A. JSON
    • B. JSON or YAML
    • C. XML
    • D. YAML

    Top

    Q4: Part of your CloudFormation deployment fails due to a mis-configuration, by defaukt what will happen?

    • A. CloudFormation will rollback only the failed components
    • B. CloudFormation will rollback the entire stack
    • C. Failed component will remain available for debugging purposes
    • D. CloudFormation will ask you if you want to continue with the deployment

    Top

    Q5: You want to receive an email whenever a user pushes code to CodeCommit repository, how can you configure this?

    • A. Create a new SNS topic and configure it to poll for CodeCommit eveents. Ask all users to subscribe to the topic to receive notifications
    • B. Configure a CloudWatch Events rule to send a message to SES which will trigger an email to be sent whenever a user pushes code to the repository.
    • C. Configure Notifications in the console, this will create a CloudWatch events rule to send a notification to a SNS topic which will trigger an email to be sent to the user.
    • D. Configure a CloudWatch Events rule to send a message to SQS which will trigger an email to be sent whenever a user pushes code to the repository.


    Answer: C

    Reference: Getting Started with Amazon SNS

    Top

    Q6: Which AWS service can be used to centrally store and version control your application source code, binaries and libraries

    • A. CodeCommit
    • B. CodeBuild
    • C. CodePipeline
    • D. ElasticFileSystem


    Answer: A

    Reference: AWS CodeCommit

    Top

    Q7: You are using CloudFormation to create a new S3 bucket,
    which of the following sections would you use to define the properties of your bucket?

    • A. Conditions
    • B. Parameters
    • C. Outputs
    • D. Resources


    Answer: D

    Reference: Resources

    Top

    Q8: You are deploying a number of EC2 and RDS instances using CloudFormation. Which section of the CloudFormation template
    would you use to define these?

    • A. Transforms
    • B. Outputs
    • C. Resources
    • D. Instances


    Answer: C.
    The Resources section defines your resources you are provisioning. Outputs is used to output user defines data relating to the resources you have built and can also used as input to another CloudFormation stack. Transforms is used to reference code located in S3.

    Reference: Resources

    Top

    Q9: Which AWS service can be used to fully automate your entire release process?

    • A. CodeDeploy
    • B. CodePipeline
    • C. CodeCommit
    • D. CodeBuild


    Answer: B.
    AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates

    Reference: AWS CodePipeline

    Top

    Q10: You want to use the output of your CloudFormation stack as input to another CloudFormation stack. Which sections of the CloudFormation template would you use to help you configure this?

    • A. Outputs
    • B. Transforms
    • C. Resources
    • D. Exports


    Answer: A.
    Outputs is used to output user defines data relating to the resources you have built and can also used as input to another CloudFormation stack.

    Reference: CloudFormation Outputs

    Top

    Q11: You have some code located in an S3 bucket that you want to reference in your CloudFormation template. Which section of the template can you use to define this?

    • A. Inputs
    • B. Resources
    • C. Transforms
    • D. Files


    Answer: C.
    Transforms is used to reference code located in S3 and also specifying the use of the Serverless Application Model (SAM)
    for Lambda deployments.
    Transform:
    Name: ‘AWS::Include’
    Parameters:
    Location: ‘s3://MyAmazonS3BucketName/MyFileName.yaml’

    Reference: Transforms

    Top

    Q12: You are deploying an application to a number of Ec2 instances using CodeDeploy. What is the name of the file
    used to specify source files and lifecycle hooks?

    • A. buildspec.yml
    • B. appspec.json
    • C. appspec.yml
    • D. buildspec.json


    Answer: C.

    Reference: CodeDeploy AppSpec File Reference

    Top

    Q13: Which of the following approaches allows you to re-use pieces of CloudFormation code in multiple templates, for common use cases like provisioning a load balancer or web server?

    • A. Share the code using an EBS volume
    • B. Copy and paste the code into the template each time you need to use it
    • C. Use a cloudformation nested stack
    • D. Store the code you want to re-use in an AMI and reference the AMI from within your CloudFormation template.


    Answer: C.

    Reference: Working with Nested Stacks

    Top

    Q14: In the CodeDeploy AppSpec file, what are hooks used for?

    • A. To reference AWS resources that will be used during the deployment
    • B. Hooks are reserved for future use
    • C. To specify files you want to copy during the deployment.
    • D. To specify, scripts or function that you want to run at set points in the deployment lifecycle


    Answer: D.
    The ‘hooks’ section for an EC2/On-Premises deployment contains mappings that link deployment lifecycle event hooks to one or more scripts.

    Reference: AppSpec ‘hooks’ Section

    Top

    Q15:You need to setup a RESTful API service in AWS that would be serviced via the following url https://democompany.com/customers Which of the following combination of services can be used for development and hosting of the RESTful service? Choose 2 answers from the options below

    • A. AWS Lambda and AWS API gateway
    • B. AWS S3 and Cloudfront
    • C. AWS EC2 and AWS Elastic Load Balancer
    • D. AWS SQS and Cloudfront

    Answer: A and C
    AWS Lambda can be used to host the code and the API gateway can be used to access the API’s which point to AWS Lambda Alternatively you can create your own API service , host it on an EC2 Instance and then use the AWS Application Load balancer to do path based routing.
    Reference: Build a Serverless Web Application with AWS Lambda, Amazon API Gateway, Amazon S3, Amazon DynamoDB, and Amazon Cognito

    Top

    Q16: As a developer, you have created a Lambda function that is used to work with a bucket in Amazon S3. The Lambda function is not working as expected. You need to debug the issue and understand what’s the underlying issue. How can you accomplish this in an easily understandable way?

    • A. Use AWS Cloudwatch metrics
    • B. Put logging statements in your code
    • C. Set the Lambda function debugging level to verbose
    • D. Use AWS Cloudtrail logs

    Answer: B
    You can insert logging statements into your code to help you validate that your code is working as expected. Lambda automatically integrates with Amazon CloudWatch Logs and pushes all logs from your code to a CloudWatch Logs group associated with a Lambda function (/aws/lambda/).
    Reference: Using Amazon CloudWatch

    Top

    Q17: You have a lambda function that is processed asynchronously. You need a way to check and debug issues if the function fails? How could you accomplish this?

    • A. Use AWS Cloudwatch metrics
    • B. Assign a dead letter queue
    • C. Congure SNS notications
    • D. Use AWS Cloudtrail logs

    Answer: B
    Any Lambda function invoked asynchronously is retried twice before the event is discarded. If the retries fail and you’re unsure why, use Dead Letter Queues (DLQ) to direct unprocessed events to an Amazon SQS queue or an Amazon SNS topic to analyze the failure.
    Reference: AWS Lambda Function Dead Letter Queues

    Top

    Q18: You are developing an application that is going to make use of Amazon Kinesis. Due to the high throughput , you decide to have multiple shards for the streams. Which of the following is TRUE when it comes to processing data across multiple shards?

    • A. You cannot guarantee the order of data across multiple shards. Its possible only within a shard
    • B. Order of data is possible across all shards in a streams
    • C. Order of data is not possible at all in Kinesis streams
    • D. You need to use Kinesis firehose to guarantee the order of data

    Answer: A
    Kinesis Data Streams lets you order records and read and replay records in the same order to many Kinesis Data Streams applications. To enable write ordering, Kinesis Data Streams expects you to call the PutRecord API to write serially to a shard while using the sequenceNumberForOrdering parameter. Setting this parameter guarantees strictly increasing sequence numbers for puts from the same client and to the same partition key.
    Option A is correct as it cannot guarantee the ordering of records across multiple shards.
    Reference: How to perform ordered data replication between applications by using Amazon DynamoDB Streams

    Top

    Q19: You’ve developed a Lambda function and are now in the process of debugging it. You add the necessary print statements in the code to assist in the debugging. You go to Cloudwatch logs , but you see no logs for the lambda function. Which of the following could be the underlying issue for this?

    • A. You’ve not enabled versioning for the Lambda function
    • B. The IAM Role assigned to the Lambda function does not have the necessary permission to create Logs
    • C. There is not enough memory assigned to the function
    • D. There is not enough time assigned to the function


    Answer: B
    “If your Lambda function code is executing, but you don’t see any log data being generated after several minutes, this could mean your execution role for the Lambda function did not grant permissions to write log data to CloudWatch Logs. For information about how to make sure that you have set up the execution role correctly to grant these permissions, see Manage Permissions: Using an IAM Role (Execution Role)”.

    Reference: Using Amazon CloudWatch

    Top

    Q20: Your application is developed to pick up metrics from several servers and push them off to Cloudwatch. At times , the application gets client 429 errors. Which of the following can be done from the programming side to resolve such errors?

    • A. Use the AWS CLI instead of the SDK to push the metrics
    • B. Ensure that all metrics have a timestamp before sending them across
    • C. Use exponential backoff in your request
    • D. Enable encryption for the requests

    Answer: C.
    The main reason for such errors is that throttling is occurring when many requests are sent via API calls. The best way to mitigate this is to stagger the rate at which you make the API calls.
    In addition to simple retries, each AWS SDK implements exponential backoff algorithm for better flow control. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses. You should implement a maximum delay interval, as well as a maximum number of retries. The maximum delay interval and maximum number of retries are not necessarily fixed values and should be set based on the operation being performed, as well as other local factors, such as network latency.
    Reference: Error Retries and Exponential Backoff in AWS

    Q21: You have been instructed to use the CodePipeline service for the CI/CD automation in your company. Due to security reasons , the resources that would be part of the deployment are placed in another account. Which of the following steps need to be carried out to accomplish this deployment? Choose 2 answers from the options given below

    • A. Dene a customer master key in KMS
    • B. Create a reference Code Pipeline instance in the other account
    • C. Add a cross account role
    • D. Embed the access keys in the codepipeline process

    Answer: A. and C.
    You might want to create a pipeline that uses resources created or managed by another AWS account. For example, you might want to use one account for your pipeline and another for your AWS CodeDeploy resources. To do so, you must create a AWS Key Management Service (AWS KMS) key to use, add the key to the pipeline, and set up account policies and roles to enable cross-account access.
    Reference: Create a Pipeline in CodePipeline That Uses Resources from Another AWS Account

    Top

    Q22: You are planning on deploying an application to the worker role in Elastic Beanstalk. Moreover, this worker application is going to run the periodic tasks. Which of the following is a must have as part of the deployment?

    • A. An appspec.yaml file
    • B. A cron.yaml  file
    • C. A cron.cong file
    • D. An appspec.json file


    Answer: B.
    Create an Application Source Bundle
    When you use the AWS Elastic Beanstalk console to deploy a new application or an application version, you’ll need to upload a source bundle. Your source bundle must meet the following requirements:
    Consist of a single ZIP file or WAR file (you can include multiple WAR files inside your ZIP file)
    Not exceed 512 MB
    Not include a parent folder or top-level directory (subdirectories are fine)
    If you want to deploy a worker application that processes periodic background tasks, your application source bundle must also include a cron.yaml file. For more information, see Periodic Tasks.

    Reference: Create an Application Source Bundle

    Top

    Q23: An application needs to make use of an SQS queue for working with messages. An SQS queue has been created with the default settings. The application needs 60 seconds to process each message. Which of the following step need to be carried out by the application.

    • A. Change the VisibilityTimeout for each message and then delete the message after processing is completed
    • B. Delete the message and change the visibility timeout.
    • C. Process the message , change the visibility timeout. Delete the message
    • D. Process the message and delete the message

    Answer: A
    If the SQS queue is created with the default settings , then the default visibility timeout is 30 seconds. And since the application needs more time for processing , you first need to change the timeout and delete the message after it is processed.
    Reference: Amazon SQS Visibility Timeout

    Top

    Q24: AWS CodeDeploy deployment fails to start & generate following error code, ”HEALTH_CONSTRAINTS_INVALID”, Which of the following can be used to eliminate this error?

    • A. Make sure the minimum number of healthy instances is equal to the total number of instances in the deployment group.
    • B. Increase the number of healthy instances required during deployment
    • C. Reduce number of healthy instances required during deployment
    • D. Make sure the number of healthy instances is equal to the specified minimum number of healthy instances.

    Answer: C
    AWS CodeDeploy generates ”HEALTH_CONSTRAINTS_INVALID” error, when a minimum number of healthy instances defined in deployment group are not available during deployment. To mitigate this error, make sure required number of healthy instances are available during deployments.
    Reference: Error Codes for AWS CodeDeploy

    Top

    Q25: How are the state machines in AWS Step Functions defined?

    • A. SAML
    • B. XML
    • C. YAML
    • D. JSON

    Answer: D. JSON
    AWS Step Functions state machines are defines in JSON files!
    Reference: What Is AWS Step Functions?

    Top

    Q26:How can API Gateway methods be configured to respond to requests?

    • A. Forwarded to method handlers
    • B. AWS Lambda
    • C. Integrated with other AWS Services
    • D. Existing HTTP endpoints

    Answer: B. C. D.

    Reference: Set up REST API Methods in API Gateway

    Top

    Q27: Which of the following could be an example of an API Gateway Resource URL for a trucks resource?

    • A. https://1a2sb3c4.execute-api.us-east-1.awsapigateway.com/trucks
    • B. https://trucks.1a2sb3c4.execute-api.us-east-1.amazonaws.com
    • C. https://1a2sb3c4.execute-api.amazonaws.com/trucks
    • D. https://1a2sb3c4.execute-api.us-east-1.amazonaws.com/cars

    Answer: C

    Reference: Amazon API Gateway Concepts

    Top

    Q28: API Gateway Deployments are:

    • A. A specific snapshot of your API’s methods
    • B. A specific snapshot of all of your API’s settings, resources, and methods
    • C. A specific snapshot of your API’s resources
    • D. A specific snapshot of your API’s resources and methods

    Answer: D.
    AWS API Gateway Deployments are a snapshot of all the resources and methods of your API and their configuration.
    Reference: Deploying a REST API in Amazon API Gateway

    Top

    Q29: A SWF workflow task or task execution can live up to how long?

    • A. 1 Year
    • B. 14 days
    • C. 24 hours
    • D. 3 days

    Answer: A. 1 Year
    Each workflow execution can run for a maximum of 1 year. Each workflow execution history can grow up to 25,000 events. If your use case requires you to go beyond these limits, you can use features Amazon SWF provides to continue executions and structure your applications using child workflow executions.
    Reference: Amazon SWF FAQs

    Top

    Q30: With AWS Step Functions, all the work in your state machine is done by tasks. These tasks performs work by using what types of things? (Choose the best 3 answers)

    • A. An AWS Lambda Function Integration
    • B. Passing parameters to API actions of other services
    • C. Activities
    • D. An EC2 Integration

    Answer: A. B. C.

    Reference:

    Top

    Q31: How does SWF make decisions?

    • A. A decider program that is written in the language of the developer’s choice
    • B. A visual workflow created in the SWF visual workflow editor
    • C. A JSON-defined state machine that contains states within it to select the next step to take
    • D. SWF outsources all decisions to human deciders through the AWS Mechanical Turk service.

    Answer: A.
    SWF allows the developer to write their own application logic to make decisions and determine how to evaluate incoming data.
    Q: What programming conveniences does Amazon SWF provide to write applications? Like other AWS services, Amazon SWF provides a core SDK for the web service APIs. Additionally, Amazon SWF offers an SDK called the AWS Flow Framework that enables you to develop Amazon SWF-based applications quickly and easily. AWS Flow Framework abstracts the details of task-level coordination with familiar programming constructs. While running your program, the framework makes calls to Amazon SWF, tracks your program’s execution state using the execution history kept by Amazon SWF, and invokes the relevant portions of your code at the right times. By offering an intuitive programming framework to access Amazon SWF, AWS Flow Framework enables developers to write entire applications as asynchronous interactions structured in a workflow. For more details, please see What is the AWS Flow Framework?
    Reference:

    Top

    Q32: In order to effectively build and test your code, AWS CodeBuild allows you to:

    • A. Select and use some 3rd party providers to run tests against your code
    • B. Select a pre-configured environment
    • C. Provide your own custom AMI
    • D. Provide your own custom container image

    Answer:A. B. and D.

    Reference: AWS CodeBuild FAQs

    Top

    Q33: X-Ray Filter Expressions allow you to search through request information using characteristics like:

    • A. URL Paths
    • B. Metadata
    • C. Trace ID
    • D. Annotations

    Top

    Q34: CodePipeline pipelines are workflows that deal with stages, actions, transitions, and artifacts. Which of the following statements is true about these concepts?

    • A. Stages contain at least two actions
    • B. Artifacts are never modified or iterated on when used inside of CodePipeline
    • C. Stages contain at least one action
    • D. Actions will have a deployment artifact as either an input an output or both

    Answer: B. C. D.

    Reference:

    Top

    Q35: When deploying a simple Python web application with Elastic Beanstalk which of the following AWS resources will be created and managed for you by Elastic Beanstalk?

    • A. An Elastic Load Balancer
    • B. An S3 Bucket
    • C. A Lambda Function
    • D. An EC2 instance

    Answer: A. B. and D.
    AWS Elastic Beanstalk uses proven AWS features and services, such as Amazon EC2, Amazon RDS, Elastic Load Balancing, Auto Scaling, Amazon S3, and Amazon SNS, to create an environment that runs your application. The current version of AWS Elastic Beanstalk uses the Amazon Linux AMI or the Windows Server 2012 R2 AMI.
    Reference: AWS Elastic Beanstalk FAQs

    Top

    Q36: Elastic Beanstalk is used to:

    • A. Deploy and scale web applications and services developed with a supported platform
    • B. Deploy and scale serverless applications
    • C. Deploy and scale applications based purely on EC2 instances
    • D. Manage the deployment of all AWS infrastructure resources of your AWS applications

    Answer: A.
    Who should use AWS Elastic Beanstalk?
    Those who want to deploy and manage their applications within minutes in the AWS Cloud. You don’t need experience with cloud computing to get started. AWS Elastic Beanstalk supports Java, .NET, PHP, Node.js, Python, Ruby, Go, and Docker web applications.
    Reference:

    Top

    Q35: How can AWS X-Ray determine what data to collect?

    • A. X-Ray applies a sampling algorithm by default
    • B. X-Ray collects data on all requests by default
    • C. You can implement your own sampling frequencies for data collection
    • D. X-Ray collects data on all requests for services enabled with it

    Answer: A. and C.

    Reference: AWS X-Ray FAQs

    Top

    Q37: Which API call is used to list all resources that belong to a CloudFormation Stack?

    • A. DescribeStacks
    • B. GetTemplate
    • C. DescribeStackResources
    • D. ListStackResources


    Answer: D.

    Reference: ListStackResources

    Top

    Q38: What is the default behaviour of a CloudFormation stack if the creation of one resource fails?

    • A. Rollback
    • B. The stack continues creating and the failed resource is ignored
    • C. Delete
    • D. Undo


    Answer: A. Rollback

    Reference: AWS CloudFormation FAQs

    Top

    Q39: Which AWS CLI command lists all current stacks in your CloudFormation service?

    • A. aws cloudformation describe-stacks
    • B. aws cloudformation list-stacks
    • C. aws cloudformation create-stack
    • D. aws cloudformation describe-stack-resources


    Answer: A. and B.

    Reference: list-stacks

    Top

    Q40:
    Which API call is used to list all resources that belong to a CloudFormation Stack?

    • A. DescribeStacks
    • B. GetTemplate
    • C. ListStackResources
    • D. DescribeStackResources


    Answer: C.

    Reference: list-stack-resources

    Top

    Q41: How does using ElastiCache help to improve database performance?

    • A. It can store petabytes of data
    • B. It provides faster internet speeds
    • C. It can store the results of frequent or highly-taxing queries
    • D. It uses read replicas

    Answer: C.
    With ElastiCache, customers get all of the benefits of a high-performance, in-memory cache with less of the administrative burden involved in launching and managing a distributed cache. The service makes setup, scaling, and cluster failure handling much simpler than in a self-managed cache deployment.
    Reference: Amazon ElastiCache

    Top

    Q42: Which of the following best describes the Lazy Loading caching strategy?

    • A. Every time the underlying database is written to or updated the cache is updated with the new information.
    • B. Every miss to the cache is counted and when a specific number is reached a full copy of the database is migrated to the cache
    • C. A specific amount of time is set before the data in the cache is marked as expired. After expiration, a request for expired data will be made through to the backing database.
    • D. Data is added to the cache when a cache miss occurs (when there is no data in the cache and the request must go to the database for that data)

    Answer: D.
    Amazon ElastiCache is an in-memory key/value store that sits between your application and the data store (database) that it accesses. Whenever your application requests data, it first makes the request to the ElastiCache cache. If the data exists in the cache and is current, ElastiCache returns the data to your application. If the data does not exist in the cache, or the data in the cache has expired, your application requests the data from your data store which returns the data to your application. Your application then writes the data received from the store to the cache so it can be more quickly retrieved next time it is requested.
    Reference: Lazy Loading

    Top

    Q43: What are two benefits of using RDS read replicas?

    • A. You can add/remove read replicas based on demand, so it creates elasticity for RDS.
    • B. Improves performance of the primary database by taking workload from it
    • C. Automatic failover in the case of Availability Zone service failures
    • D. Allows both reads and writes

    Answer: A. and B.

    Reference: Amazon RDS Read Replicas

    Top

    Q44: What is the simplest way to enable an S3 bucket to be able to send messages to your SNS topic?

    • A. Attach an IAM role to the S3 bucket to send messages to SNS.
    • B. Activate the S3 pipeline feature to send notifications to another AWS service – in this case select SNS.
    • C. Add a resource-based access control policy on the SNS topic.
    • D. Use AWS Lambda to receive events from the S3 bucket and then use the Publish API action to send them to the SNS topic.

    Top

    Q45: You have just set up a push notification service to send a message to an app installed on a device with the Apple Push Notification Service. It seems to work fine. You now want to send a message to an app installed on devices for multiple platforms, those being the Apple Push Notification Service(APNS) and Google Cloud Messaging for Android (GCM). What do you need to do first for this to be successful?

    • A. Request Credentials from Mobile Platforms, so that each device has the correct access control policies to access the SNS publisher
    • B. Create a Platform Application Object which will connect all of the mobile devices with your app to the correct SNS topic.
    • C. Request a Token from Mobile Platforms, so that each device has the correct access control policies to access the SNS publisher.
    • D. Get a set of credentials in order to be able to connect to the push notification service you are trying to setup.

    Answer: D.
    To use Amazon SNS mobile push notifications, you need to establish a connection with a supported push notification service. This connection is established using a set of credentials.
    Reference: Add Device Tokens or Registration IDs

    Top

    Q46: SNS message can be sent to different kinds of endpoints. Which of these is NOT currently a supported endpoint?

    • A. Slack Messages
    • B. SMS (text message)
    • C. HTTP/HTTPS
    • D. AWS Lambda

    Answer: A.
    Slack messages are not directly integrated with SNS, though theoretically, you could write a service to push messages to slack from SNS.
    Reference:

    Top

    Q47: Company B provides an online image recognition service and utilizes SQS to decouple system components for scalability. The SQS consumers poll the imaging queue as often as possible to keep end-to-end throughput as high as possible. However, Company B is realizing that polling in tight loops is burning CPU cycles and increasing costs with empty responses. How can Company B reduce the number empty responses?

    • A. Set the imaging queue VisibilityTimeout attribute to 20 seconds
    • B. Set the imaging queue MessageRetentionPeriod attribute to 20 seconds
    • C. Set the imaging queue ReceiveMessageWaitTimeSeconds Attribute to 20 seconds
    • D. Set the DelaySeconds parameter of a message to 20 seconds

    Answer: C.
    Enabling long polling reduces the amount of false and empty responses from SQS service. It also reduces the number of calls that need to be made to a queue by staying connected to the queue until all messages have been received or until timeout. In order to enable long polling the ReceiveMessageWaitTimeSeconds attribute needs to be set to a number greater than 0. If it is set to 0 then short polling is enabled.
    Reference: Amazon SQS Long Polling

    Top

    Q48: Which of the following statements about SQS standard queues are true?

    • A. Message order can be indeterminate – you’re not guaranteed to get messages in the same order they were sent in
    • B. Messages will be delivered exactly once and messages will be delivered in First in, First out order
    • C. Messages will be delivered exactly once and message delivery order is indeterminate
    • D. Messages can be delivered one or more times

    Answer: A. and D.
    A standard queue makes a best effort to preserve the order of messages, but more than one copy of a message might be delivered out of order. If your system requires that order be preserved, we recommend using a FIFO (First-In-First-Out) queue or adding sequencing information in each message so you can reorder the messages when they’re received.
    Reference: Amazon SQS Standard Queues

    Top

    Q49: Which of the following is true if long polling is enabled?

    • A. If long polling is enabled, then each poll only polls a subset of SQS servers; in order for all messages to be received, polling must continuously occur
    • B. The reader will listen to the queue until timeout
    • C. Increases costs because each request lasts longer
    • D. The reader will listen to the queue until a message is available or until timeout

    Answer: D.

    Reference: Amazon SQS Long Polling

    Top

    Q50: When dealing with session state in EC2-based applications using Elastic load balancers which option is generally thought of as the best practice for managing user sessions?

    • A. Having the ELB distribute traffic to all EC2 instances and then having the instance check a caching solution like ElastiCache running Redis or Memcached for session information
    • B. Permanently assigning users to specific instances and always routing their traffic to those instances
    • C. Using Application-generated cookies to tie a user session to a particular instance for the cookie duration
    • D. Using Elastic Load Balancer generated cookies to tie a user session to a particular instance

    Answer: A.

    Reference: Distributed Session Management

    Top

    Q51: When requested through an STS API call, credentials are returned with what three components?

    • A. Security Token, Access Key ID, Signed URL
    • B. Security Token, Access Key ID, Secret Access Key
    • C. Signed URL, Security Token, Username
    • D. Security Token, Secret Access Key, Personal Pin Code

    Answer: B.
    Security Token, Access Key ID, Secret Access Key
    Reference:

    Top

    Q52: Your application must write to an SQS queue. Your corporate security policies require that AWS credentials are always encrypted and are rotated at least once a week.
    How can you securely provide credentials that allow your application to write to the queue?

    • A. Have the application fetch an access key from an Amazon S3 bucket at run time.
    • B. Launch the application’s Amazon EC2 instance with an IAM role.
    • C. Encrypt an access key in the application source code.
    • D. Enroll the instance in an Active Directory domain and use AD authentication.

    Answer: B.
    IAM roles are based on temporary security tokens, so they are rotated automatically. Keys in the source code cannot be rotated (and are a very bad idea). It’s impossible to retrieve credentials from an S3 bucket if you don’t already have credentials for that bucket. Active Directory authorization will not grant access to AWS resources.
    Reference: AWS IAM FAQs

    Top

    Q53: Your web application reads an item from your DynamoDB table, changes an attribute, and then writes the item back to the table. You need to ensure that one process doesn’t overwrite a simultaneous change from another process.
    How can you ensure concurrency?

    • A. Implement optimistic concurrency by using a conditional write.
    • B. Implement pessimistic concurrency by using a conditional write.
    • C. Implement optimistic concurrency by locking the item upon read.
    • D. Implement pessimistic concurrency by locking the item upon read.

    Answer: A.
    Optimistic concurrency depends on checking a value upon save to ensure that it has not changed. Pessimistic concurrency prevents a value from changing by locking the item or row in the database. DynamoDB does not support item locking, and conditional writes are perfect for implementing optimistic concurrency.
    Reference: Optimistic Locking With Version Number

    Top

    Q54: Which statements about DynamoDB are true? Choose 2 answers

    • A. DynamoDB uses optimistic concurrency control
    • B. DynamoDB restricts item access during writes
    • C. DynamoDB uses a pessimistic locking model
    • D. DynamoDB restricts item access during reads
    • E. DynamoDB uses conditional writes for consistency


    Top

    Q55: Your CloudFormation template has the following Mappings section:

    Which JSON snippet will result in the value “ami-6411e20d” when a stack is launched in us-east-1?

    • A. { “Fn::FindInMap” : [ “Mappings”, { “RegionMap” : [“us-east-1”, “us-west-1”] }, “32”]}
    • B. { “Fn::FindInMap” : [ “Mappings”, { “Ref” : “AWS::Region” }, “32”]}
    • C. { “Fn::FindInMap” : [ “RegionMap”, { “Ref” : “AWS::Region” }, “32”]}
    • D. { “Fn::FindInMap” : [ “RegionMap”, { “RegionMap” : “AWS::Region” }, “32”]}

    Answer: C.
    The intrinsic function Fn::FindInMap returns the value corresponding to keys in a two-level map that is declared in the Mappings section.
    You can use the Fn::FindInMap function to return a named value based on a specified key. The following example template contains an Amazon EC2 resource whose ImageId property is assigned by the FindInMap function. The FindInMap function specifies key as the region where the stack is created (using the AWS::Region pseudo parameter) and HVM64 as the name of the value to map to.
    Reference:

    Top

    Q56: Your application triggers events that must be delivered to all your partners. The exact partner list is constantly changing: some partners run a highly available endpoint, and other partners’ endpoints are online only a few hours each night. Your application is mission-critical, and communication with your partners must not introduce delay in its operation. A delay in delivering the event to one partner cannot delay delivery to other partners.

    What is an appropriate way to code this?

    • A. Implement an Amazon SWF task to deliver the message to each partner. Initiate an Amazon SWF workflow execution.
    • B. Send the event as an Amazon SNS message. Instruct your partners to create an HTTP. Subscribe their HTTP endpoint to the Amazon SNS topic.
    • C. Create one SQS queue per partner. Iterate through the queues and write the event to each one. Partners retrieve messages from their queue.
    • D. Send the event as an Amazon SNS message. Create one SQS queue per partner that subscribes to the Amazon SNS topic. Partners retrieve messages from their queue.

    Answer: D.
    There are two challenges here: the command must be “fanned out” to a variable pool of partners, and your app must be decoupled from the partners because they are not highly available.
    Sending the command as an SNS message achieves the fan-out via its publication/subscribe model, and using an SQS queue for each partner decouples your app from the partners. Writing the message to each queue directly would cause more latency for your app and would require your app to monitor which partners were active. It would be difficult to write an Amazon SWF workflow for a rapidly changing set of partners.

    Reference: AWS SNS Faqs

    Top

    Q57: You have a three-tier web application (web, app, and data) in a single Amazon VPC. The web and app tiers each span two Availability Zones, are in separate subnets, and sit behind ELB Classic Load Balancers. The data tier is a Multi-AZ Amazon RDS MySQL database instance in database subnets.
    When you call the database tier from your app tier instances, you receive a timeout error. What could be causing this?

    • A. The IAM role associated with the app tier instances does not have rights to the MySQL database.
    • B. The security group for the Amazon RDS instance does not allow traffic on port 3306 from the app
      instances.
    • C. The Amazon RDS database instance does not have a public IP address.
    • D. There is no route defined between the app tier and the database tier in the Amazon VPC.

    Answer: B.
    Security groups block all network traffic by default, so if a group is not correctly configured, it can lead to a timeout error. MySQL security, not IAM, controls MySQL security. All subnets in an Amazon VPC have routes to all other subnets. Internal traffic within an Amazon VPC does not require public IP addresses.

    Reference: Security Groups for Your VPC

    Top

    Q58: What type of block cipher does Amazon S3 offer for server side encryption?

    • A. RC5
    • B. Blowfish
    • C. Triple DES
    • D. Advanced Encryption Standard

    Answer: D
    Amazon S3 server-side encryption uses one of the strongest block ciphers available, 256-bit Advanced Encryption Standard (AES-256), to encrypt your data.

    Reference: Protecting Data Using Server-Side Encryption

    Top

    Q59: You have written an application that uses the Elastic Load Balancing service to spread
    traffic to several web servers Your users complain that they are sometimes forced to login
    again in the middle of using your application, after they have already togged in. This is not
    behaviour you have designed. What is a possible solution to prevent this happening?

    • A. Use instance memory to save session state.
    • B. Use instance storage to save session state.
    • C. Use EBS to save session state
    • D. Use ElastiCache to save session state.
    • E. Use Glacier to save session slate.

    Answer: D.
    You can cache a variety of objects using the service, from the content in persistent data stores (such as Amazon RDS, DynamoDB, or self-managed databases hosted on EC2) to dynamically generated web pages (with Nginx for example), or transient session data that may not require a persistent backing store. You can also use it to implement high-frequency counters to deploy admission control in high volume web applications.

    Reference: Amazon ElastiCache FAQs

    Top

    Q60: You are writing to a DynamoDB table and receive the following exception:”
    ProvisionedThroughputExceededException”. though according to your Cloudwatch metrics
    for the table, you are not exceeding your provisioned throughput. What could be an
    explanation for this?

    • A. You haven’t provisioned enough DynamoDB storage instances
    • B. You’re exceeding your capacity on a particular Range Key
    • C. You’re exceeding your capacity on a particular Hash Key
    • D. You’re exceeding your capacity on a particular Sort Key
    • E. You haven’t configured DynamoDB Auto Scaling triggers

    Answer: C.
    The primary key that uniquely identifies each item in a DynamoDB table can be simple (a partition key only) or composite (a partition key combined with a sort key).
    Generally speaking, you should design your application for uniform activity across all logical partition keys in the Table and its secondary indexes.
    You can determine the access patterns that your application requires, and estimate the total read capacity units and write capacity units that each table and secondary Index requires.

    As traffic starts to flow, DynamoDB automatically supports your access patterns using the throughput you have provisioned, as long as the traffic against a given partition key does not exceed 3000 read capacity units or 1000 write capacity units.

    Reference: Best Practices for Designing and Using Partition Keys Effectively

    Top

    Q61: Which DynamoDB limits can be raised by contacting AWS support?

    • A. The number of hash keys per account
    • B. The maximum storage used per account
    • C. The number of tables per account
    • D. The number of local secondary indexes per account
    • E. The number of provisioned throughput units per account


    Answer: C. and E.

    For any AWS account, there is an initial limit of 256 tables per region.
    AWS places some default limits on the throughput you can provision.
    These are the limits unless you request a higher amount.
    To request a service limit increase see https://aws.amazon.com/support.Reference: Limits in DynamoDB


    Top

    Q62: AWS CodeBuild allows you to compile your source code, run unit tests, and produce deployment artifacts by:

    • A. Allowing you to provide an Amazon Machine Image to take these actions within
    • B. Allowing you to select an Amazon Machine Image and provide a User Data bootstrapping script to prepare an instance to take these actions within
    • C. Allowing you to provide a container image to take these actions within
    • D. Allowing you to select from pre-configured environments to take these actions within

    Answer: C. and D.
    You can provide your own custom container image to build your deployment artifacts.
    You never actually pass a specific AMI to CodeBuild. Though you can provide a custom docker image which you could basically ‘bootstrap’ for the purposes of your build.
    Reference: AWS CodeBuild Faqs

    Top

    Q63: Which of the following will not cause a CloudFormation stack deployment to rollback?

    • A. The template contains invalid JSON syntax
    • B. An AMI specified in the template exists in a different region than the one in which the stack is being deployed.
    • C. A subnet specified in the template does not exist
    • D. The template specifies an instance-store backed AMI and an incompatible EC2 instance type.

    Answer: A.
    Invalid JSON syntax will cause an error message during template validation. Until the syntax is fixed, the template will not be able to deploy resources, so there will not be a need to or opportunity to rollback.
    Reference: AWS CloudFormatio Faqs

    Top

    Q64: Your team is using CodeDeploy to deploy an application which uses secure parameters that are stored in the AWS System Mangers Parameter Store. What two options below must be completed so CodeDeploy can deploy the application?

    • A. Use ssm get-parameters with –with-decryption option
    • B. Add permissions using AWS access keys
    • C. Add permissions using AWS IAM role
    • D. Use ssm get-parameters with –with-no-decryption option

    Answer: A. and C.

    Reference: Add permission using IAM role


    Top

    Q65: A corporate web application is deployed within an Amazon VPC, and is connected to the corporate data center via IPSec VPN. The application must authenticate against the on-premise LDAP server. Once authenticated, logged-in users can only access an S3 keyspace specific to the user. Which of the solutions below meet these requirements? Choose two answers How would you authenticate to the application given these details? (Choose 2)

    • A. The application authenticates against LDAP, and retrieves the name of an IAM role associated with the user. The application then calls the IAM Security Token Service to assume that IAM Role. The application can use the temporary credentials to access the S3 keyspace.
    • B. Develop an identity broker which authenticates against LDAP, and then calls IAM Security Token Service to get IAM federated user credentials. The application calls the identity broker to get IAM federated user credentials with access to the appropriate S3 keyspace
    • C. Develop an identity broker which authenticates against IAM Security Token Service to assume an IAM Role to get temporary AWS security credentials. The application calls the identity broker to get AWS temporary security credentials with access to the app
    • D. The application authenticates against LDAP. The application then calls the IAM Security Service to login to IAM using the LDAP credentials. The application can use the IAM temporary credentials to access the appropriate S3 bucket.

    Answer: A. and B.
    The question clearly says “authenticate against LDAP”. Temporary credentials come from STS. Federated user credentials come from the identity broker.
    Reference: IAM faqs

    Top

    Q66:
    A corporate web application is deployed within an Amazon VPC, and is connected to the corporate data center via IPSec VPN. The application must authenticate against the on-premise LDAP server. Once authenticated, logged-in users can only access an S3 keyspace specific to the user. Which of the solutions below meet these requirements? Choose two answers
    How would you authenticate to the application given these details? (Choose 2)

    • A. The application authenticates against LDAP, and retrieves the name of an IAM role associated with the user. The application then calls the IAM Security Token Service to assume that IAM Role. The application can use the temporary credentials to access the S3 keyspace.
    • B. Develop an identity broker which authenticates against LDAP, and then calls IAM Security Token Service to get IAM federated user credentials. The application calls the identity broker to get IAM federated user credentials with access to the appropriate S3 keyspace
    • C. Develop an identity broker which authenticates against IAM Security Token Service to assume an IAM Role to get temporary AWS security credentials. The application calls the identity broker to get AWS temporary security credentials with access to the app
    • D. The application authenticates against LDAP. The application then calls the IAM Security Service to login to IAM using the LDAP credentials. The application can use the IAM temporary credentials to access the appropriate S3 bucket.

    Answer: A. and B.
    The question clearly says “authenticate against LDAP”. Temporary credentials come from STS. Federated user credentials come from the identity broker.
    Reference: AWA STS Faqs

    Top

    Q67: When users are signing in to your application using Cognito, what do you need to do to make sure if the user has compromised credentials, they must enter a new password?

    • A. Create a user pool in Cognito
    • B. Block use for “Compromised credential” in the Basic security section
    • C. Block use for “Compromised credential” in the Advanced security section
    • D. Use secure remote password

    Answer: A. and C.
    Amazon Cognito can detect if a user’s credentials (user name and password) have been compromised elsewhere. This can happen when users reuse credentials at more than one site, or when they use passwords that are easy to guess.

    From the Advanced security page in the Amazon Cognito console, you can choose whether to allow, or block the user if compromised credentials are detected. Blocking requires users to choose another password. Choosing Allow publishes all attempted uses of compromised credentials to Amazon CloudWatch. For more information, see Viewing Advanced Security Metrics.

    You can also choose whether Amazon Cognito checks for compromised credentials during sign-in, sign-up, and password changes.

    Note Currently, Amazon Cognito doesn’t check for compromised credentials for sign-in operations with Secure Remote Password (SRP) flow, which doesn’t send the password during sign-in. Sign-ins that use the AdminInitiateAuth API with ADMIN_NO_SRP_AUTH flow and the InitiateAuth API with USER_PASSWORD_AUTH flow are checked for compromised credentials.

    Reference: AWS Cognito

    Top

    Q68: You work in a large enterprise that is currently evaluating options to migrate your 27 GB Subversion code base. Which of the following options is the best choice for your organization?

    • A. AWS CodeHost
    • B. AWS CodeCommit
    • C. AWS CodeStart
    • D. None of these

    Answer: D.
    None of these. While CodeCommit is a good option for git reponsitories it is not able to host Subversion source control.

    Reference: Migration to CodeCommit

    Top

    Q69: You are on a development team and you need to migrate your Spring Application over to AWS. Your team is looking to build, modify, and test new versions of the application. What AWS services could help you migrate your app?

    • A. Elastic Beanstalk
    • B. SQS
    • C. Ec2
    • D. AWS CodeDeploy

    Answer: A. C. and D.
    Amazon EC2 can be used to deploy various applications to your AWS Infrastructure.
    AWS CodeDeploy is a deployment service that automates application deployments to Amazon EC2 instances, on-premises instances, or serverless Lambda functions.

    Reference: AWS Deployment Faqs

    Top

    Q70:
    You are a developer responsible for managing a high volume API running in your company’s datacenter. You have been asked to implement a similar API, but one that has potentially higher volume. And you must do it in the most cost effective way, using as few services and components as possible. The API stores and fetches data from a key value store. Which services could you utilize in AWS?

    • A. DynamoDB
    • B. Lambda
    • C. API Gateway
    • D. EC2

    Answer: A. and C.
    NoSQL databases like DynamoDB are designed for key value usage. DynamoDB can also handle incredible volumes and is cost effective. AWS API Gateway makes it easy for developers to create, publish, maintain, monitor, and secure APIs.

    Reference: API Gateway Faqs

    Top

    Q71: By default, what event occurs if your CloudFormation receives an error during creation?

    • A. DELETE_IN_PROGRESS
    • B. CREATION_IN_PROGRESS
    • C. DELETE_COMPLETE
    • D. ROLLBACK_IN_PROGRESS

    Answer: D.

    Reference: Check Status Code


    Top

    Q72:
    AWS X-Ray was recently implemented inside of a service that you work on. Several weeks later, after a new marketing push, that service started seeing a large spike in traffic and you’ve been tasked with investigating a few issues that have started coming up but when you review the X-Ray data you can’t find enough information to draw conclusions so you decide to:

    • A. Start passing in the X-Amzn-Trace-Id: True HTTP header from your upstream requests
    • B. Refactor the service to include additional calls to the X-Ray API using an AWS SDK
    • C. Update the sampling algorithm to increase the sample rate and instrument X-Ray to collect more pertinent information
    • D. Update your application to use the custom API Gateway TRACE method to send in data

    Answer: C.
    This is a good way to solve the problem – by customizing the sampling so that you can get more relevant information.

    Reference: AWS X-Ray facts

    Top

    Q74: X-Ray metadata:

    • A. Associates request data with a particular Trace-ID
    • B. Stores key-value pairs of any type that are not searchable
    • C. Collects at the service layer to provide information on the overall health of the system
    • D. Stores key-value pairs of searchable information

    Answer:AB.
    X-Ray metadata stores key-value pairs of any type that are not searchable.
    Reference: AWS X-Rays faqs

    Top

    Q75: Which of the following is the right sequence that gets called in CodeDeploy when you use Lambda hooks in an EC2/On-Premise Deployment?

    • A. Before Install-AfterInstall-Validate Service-Application Start
    • B. Before Install-After-Install-Application Stop-Application Start
    • C. Before Install-Application Stop-Validate Service-Application Start
    • D. Application Stop-Before Install-After Install-Application Start

    Answer: D.
    In an in-place deployment, including the rollback of an in-place deployment, event hooks are run in the following order:

    Note An AWS Lambda hook is one Lambda function specified with a string on a new line after the name of the lifecycle event. Each hook is executed once per deployment. Following are descriptions of the lifecycle events where you can run a hook during an Amazon ECS deployment.

    BeforeInstall – Use to run tasks before the replacement task set is created. One target group is associated with the original task set. If an optional test listener is specified, it is associated with the original task set. A rollback is not possible at this point. AfterInstall – Use to run tasks after the replacement task set is created and one of the target groups is associated with it. If an optional test listener is specified, it is associated with the original task set. The results of a hook function at this lifecycle event can trigger a rollback. AfterAllowTestTraffic – Use to run tasks after the test listener serves traffic to the replacement task set. The results of a hook function at this point can trigger a rollback. BeforeAllowTraffic – Use to run tasks after the second target group is associated with the replacement task set, but before traffic is shifted to the replacement task set. The results of a hook function at this lifecycle event can trigger a rollback. AfterAllowTraffic – Use to run tasks after the second target group serves traffic to the replacement task set. The results of a hook function at this lifecycle event can trigger a rollback. Run Order of Hooks in an Amazon ECS Deployment

    In an Amazon ECS deployment, event hooks run in the following order:

    For in-place deployments, the six hooks related to blocking and allowing traffic apply only if you specify a Classic Load Balancer, Application Load Balancer, or Network Load Balancer from Elastic Load Balancing in the deployment group.Note The Start, DownloadBundle, Install, and End events in the deployment cannot be scripted, which is why they appear in gray in this diagram. However, you can edit the ‘files’ section of the AppSpec file to specify what’s installed during the Install event.

    Reference: Appspec.yml specs

    Top

    Q76:
    Describe the process of registering a mobile device with SNS push notification service using GCM.

    • A. Receive Registration ID and token for each mobile device. Then, register the mobile application with Amazon SNS, and pass the GCM token credentials to Amazon SNS
    • B. Pass device token to SNS to create mobile subscription endpoint for each mobile device, then request the device token from each mobile device. SNS then communicates on your behalf to the GCM service
    • C. None of these are correct
    • D. Submit GCM notification credentials to Amazon SNS, then receive the Registration ID for each mobile device. After that, pass the device token to SNS, and SNS then creates a mobile subscription endpoint for each device and communicates with the GCM service on your behalf

    Answer: D.
    When you first register an app and mobile device with a notification service, such as Apple Push Notification Service (APNS) and Google Cloud Messaging for Android (GCM), device tokens or registration IDs are returned from the notification service. When you add the device tokens or registration IDs to Amazon SNS, they are used with the PlatformApplicationArn API to create an endpoint for the app and device. When Amazon SNS creates the endpoint, an EndpointArn is returned. The EndpointArn is how Amazon SNS knows which app and mobile device to send the notification message to.

    Reference: AWS Mobile Push Send device token

    Top

    Q77:
    You run an ad-supported photo sharing website using S3 to serve photos to visitors of your site. At some point you find out that other sites have been linking to the photos on your site, causing loss to your business. What is an effective method to mitigate this?

    • A. Store photos on an EBS volume of the web server.
    • B. Block the IPs of the offending websites in Security Groups.
    • C. Remove public read access and use signed URLs with expiry dates.
    • D. Use CloudFront distributions for static content.

    Answer: C.
    This solves the issue, but does require you to modify your website. Your website already uses S3, so it doesn’t require a lot of changes. See the docs for details: http://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html

    Reference: AWS S3 shared objects presigned urls

    CloudFront on its own doesn’t prevent unauthorized access and requires you to add a whole new layer to your stack (which may make sense anyway). You can serve private content, but you’d have to use signed URLs or similar mechanism. Here are the docs: http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html

    Top

    Q78: How can you control access to the API Gateway in your environment?

    • A. Cognito User Pools
    • B. Lambda Authorizers
    • C. API Methods
    • D. API Stages

    Answer: A. and B.
    Access to a REST API Using Amazon Cognito User Pools as Authorizer
    As an alternative to using IAM roles and policies or Lambda authorizers (formerly known as custom authorizers), you can use an Amazon Cognito user pool to control who can access your API in Amazon API Gateway.

    To use an Amazon Cognito user pool with your API, you must first create an authorizer of the COGNITO_USER_POOLS type and then configure an API method to use that authorizer. After the API is deployed, the client must first sign the user in to the user pool, obtain an identity or access token for the user, and then call the API method with one of the tokens, which are typically set to the request’s Authorization header. The API call succeeds only if the required token is supplied and the supplied token is valid, otherwise, the client isn’t authorized to make the call because the client did not have credentials that could be authorized.

    The identity token is used to authorize API calls based on identity claims of the signed-in user. The access token is used to authorize API calls based on the custom scopes of specified access-protected resources. For more information, see Using Tokens with User Pools and Resource Server and Custom Scopes.

    Reference: AWS API Gateway integrate with Cognito

    Top

    Q79: What kind of message does SNS send to endpoints?

    • A. An XML document with parameters like Message, Source, Destination, Type
    • B. A JSON document with parameters like Message, Signature, Subject, Type.
    • C. An XML document with parameters like Message, Signature, Subject, Type
    • D. A JSON document with parameters like Message, Source, Destination, Type

    Answer: B.
    Amazon SNS messages do not publish the source/destination

    Reference: AWS SNS Faqs

    Top

    Q80: Company B provides an online image recognition service and utilizes SQS to decouple system components for scalability. The SQS consumers poll the imaging queue as often as possible to keep end-to-end throughput as high as possible. However, Company B is realizing that polling in tight loops is burning CPU cycles and increasing costs with empty responses. How can Company B reduce the number of empty responses?

    • A. Set the imaging queue MessageRetentionPeriod attribute to 20 seconds.
    • B. Set the imaging queue ReceiveMessageWaitTimeSeconds attribute to 20 seconds.
    • C. Set the imaging queue VisibilityTimeout attribute to 20 seconds.
    • D. Set the DelaySeconds parameter of a message to 20 seconds.

    Answer: B.
    ReceiveMessageWaitTimeSeconds, when set to greater than zero, enables long polling. Long polling allows the Amazon SQS service to wait until a message is available in the queue before sending a response. Short polling continuously pools a queue and can have false positives. Enabling long polling reduces the number of poll requests, false positives, and empty responses.
    Reference: AWS SQS Long Polling

    Top

    81: You’re using CloudFormation templates to build out staging environments. What section of the CloudFormation would you edit in order to allow the user to specify the PEM key-name at start time?

    • A. Resources Section
    • B. Parameters Section
    • C. Mappings Section
    • D. Declaration Section


    Answer:B.

    Parameters property type in CloudFormation allows you to accept user input when starting the CloudFormation template. It allows you to reference the user input as variable throughout your CloudFormation template. Other examples might include asking the user starting the template to provide Domain admin passwords, instance size, pem key, region, and other dynamic options.

    Reference: AWS CloudFormation Parameters


    Top

    Q82: You are writing an AWS CloudFormation template and you want to assign values to properties that will not be available until runtime. You know that you can use intrinsic functions to do this but are unsure as to which part of the template they can be used in. Which of the following is correct in describing how you can currently use intrinsic functions in an AWS CloudFormation template?

    • A. You can use intrinsic functions in any part of a template, except AWSTemplateFormatVersion and Description
    • B. You can use intrinsic functions in any part of a template.
    • C. You can use intrinsic functions only in the resource properties part of a template.
    • D. You can only use intrinsic functions in specific parts of a template. You can use intrinsic functions in resource properties, metadata attributes, and update policy attributes.


    Answer: D.

    You can use intrinsic functions only in specific parts of a template. Currently, you can use intrinsic functions in resource properties, outputs, metadata attributes, and update policy attributes. You can also use intrinsic functions to conditionally create stack resources.

  1. Reference: AWS Intrinsic Functions

Top




Other AWS Facts and Summaries and Questions/Answers Dump

Latest DevOps and SysAdmin Feed

Djamgatech: AI Driven Certification Preparation App for iOS, Android, Windows

AI Dashboard is available on the Web, Apple, Google, and Microsoft, PRO version

Latest DevOps and SysAdmin Feed

DevOps is a set of practices and tools that organizations use to accelerate software development and improve the quality of their software products. It aims to bring development and operations teams together, so they can work more collaboratively and efficiently to deliver software faster and with fewer errors.

The goal of DevOps is to automate as much of the software delivery process as possible, using tools such as continuous integration, continuous delivery, and infrastructure as code. This allows teams to move faster and release new features and bug fixes more frequently, while also reducing the risk of errors and downtime.

DevOps also emphasizes the importance of monitoring, logging, and testing to ensure that software is performing well in production. By continuously monitoring and analyzing performance data, teams can quickly identify and resolve any issues that arise.

In summary, DevOps is a combination of people, processes, and technology that organizations use to improve their software delivery capabilities, increase efficiency, and reduce risk.

What is DevOps in Simple English?

What is a System Administrator?


DevOps: In IT world, DevOps means Development Operations. The DevOps is the bridge between the developers, the servers and the infrastructure and his main role is to automate the process of delivering code to operations.
DevOps on wikipedia: is a software development process that emphasizes communication and collaboration between product management, software development, and operations professionals. DevOps also automates the process of software integration, testing, deployment and infrastructure changes.[1][2] It aims to establish a culture and environment where building, testing, and releasing software can happen rapidly, frequently, and more reliably.

DevOps Latest Feeds


DevOps Resources

  1. What is DevOps? Tackling some frequently asked questions
  2. Find Remote DevOps Jobs here.
Pass the 2023 AWS Cloud Practitioner CCP CLF-C02 Certification with flying colors Ace the 2023 AWS Solutions Architect Associate SAA-C03 Exam with Confidence Pass the 2023 AWS Certified Machine Learning Specialty MLS-C01 Exam with Flying Colors

List of Freely available programming books - What is the single most influential book every Programmers should read



#BlackOwned #BlackEntrepreneurs #BlackBuniness #AWSCertified #AWSCloudPractitioner #AWSCertification #AWSCLFC02 #CloudComputing #AWSStudyGuide #AWSTraining #AWSCareer #AWSExamPrep #AWSCommunity #AWSEducation #AWSBasics #AWSCertified #AWSMachineLearning #AWSCertification #AWSSpecialty #MachineLearning #AWSStudyGuide #CloudComputing #DataScience #AWSCertified #AWSSolutionsArchitect #AWSArchitectAssociate #AWSCertification #AWSStudyGuide #CloudComputing #AWSArchitecture #AWSTraining #AWSCareer #AWSExamPrep #AWSCommunity #AWSEducation #AzureFundamentals #AZ900 #MicrosoftAzure #ITCertification #CertificationPrep #StudyMaterials #TechLearning #MicrosoftCertified #AzureCertification #TechBooks

Top 1000 Canada Quiz and trivia: CANADA CITIZENSHIP TEST- HISTORY - GEOGRAPHY - GOVERNMENT- CULTURE - PEOPLE - LANGUAGES - TRAVEL - WILDLIFE - HOCKEY - TOURISM - SCENERIES - ARTS - DATA VISUALIZATION
zCanadian Quiz and Trivia, Canadian History, Citizenship Test, Geography, Wildlife, Secenries, Banff, Tourism

Top 1000 Africa Quiz and trivia: HISTORY - GEOGRAPHY - WILDLIFE - CULTURE - PEOPLE - LANGUAGES - TRAVEL - TOURISM - SCENERIES - ARTS - DATA VISUALIZATION
Africa Quiz, Africa Trivia, Quiz, African History, Geography, Wildlife, Culture

Exploring the Pros and Cons of Visiting All Provinces and Territories in Canada.
Exploring the Pros and Cons of Visiting All Provinces and Territories in Canada

Exploring the Advantages and Disadvantages of Visiting All 50 States in the USA
Exploring the Advantages and Disadvantages of Visiting All 50 States in the USA


Health Health, a science-based community to discuss health news and the coronavirus (COVID-19) pandemic

Today I Learned (TIL) You learn something new every day; what did you learn today? Submit interesting and specific facts about something that you just found out here.

Reddit Science This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.

Reddit Sports Sports News and Highlights from the NFL, NBA, NHL, MLB, MLS, and leagues around the world.

error: Content is protected !!