Category: noSQL

Posted on

How does a database handle pagination?

How does a database handle pagination?

How does a database handle pagination?

How does a database handle pagination?

It doesn’t. First, a database is a collection of related data, so I assume you mean DBMS or database language.

Second, pagination is generally a function of the front-end and/or middleware, not the database layer.

But some database languages provide helpful facilities that aide in implementing pagination. For example, many SQL dialects provide LIMIT and OFFSET clauses that can be used to emit up to n rows starting at a given row number. I.e., a “page” of rows. If the query results are sorted via ORDER BY and are generally unchanged between successive invocations, then that can be used to implement pagination.

That may not be the most efficient or effective implementation, though.

How does a database handle pagination?

So how do you propose pagination should be done?

On context of web apps , let’s say there are 100 mn users. One cannot dump all the users in response.

Cache database query results in the middleware layer using Redis or similar and serve out pages of rows from that.

What if you have 30, 000 rows plus, do you fetch all of that from the database and cache in Redis?

I feel the most efficient solution is still offset and limit. It doesn’t make sense to use a database and then end up putting all of your data in Redis especially data that changes a lot. Redis is not for storing all of your data.

If you have large data set, you should use offset and limit, getting only what is needed from the database into main memory (and maybe caching those in Redis) at any point in time is very efficient.

With 30,000 rows in a table, if offset/limit is the only viable or appropriate restriction, then that’s sometimes the way to go.

More often, there’s a much better way of restricting 30,000 rows via some search criteria that significantly reduces the displayed volume of rows — ideally to a single page or a few pages (which are appropriate to cache in Redis.)

It’s unlikely (though it does happen) that users really want to casually browse 30,000 rows, page by page. More often, they want this one record, or these small number of records.

 

Question: This is a general question that applies to MySQL, Oracle DB or whatever else might be out there.

I know for MySQL there is LIMIT offset,size; and for Oracle there is ‘ROW_NUMBER’ or something like that.

Ace the 2024 AWS Certified Machine Learning Specialty Exam with Flying Colors

AI- Powered Jobs Interview Warmup For Job Seekers

But when such ‘paginated’ queries are called back to back, does the database engine actually do the entire ‘select’ all over again and then retrieve a different subset of results each time? Or does it do the overall fetching of results only once, keeps the results in memory or something, and then serves subsets of results from it for subsequent queries based on offset and size?

If it does the full fetch every time, then it seems quite inefficient.

If it does full fetch only once, it must be ‘storing’ the query somewhere somehow, so that the next time that query comes in, it knows that it has already fetched all the data and just needs to extract next page from it. In that case, how will the database engine handle multiple threads? Two threads executing the same query?

something will be quick or slow without taking measurements, and complicate the code in advance to download 12 pages at once and cache them because “it seems to me that it will be faster”.

Answer: First of all, do not make assumptions in advance whether something will be quick or slow without taking measurements, and complicate the code in advance to download 12 pages at once and cache them because “it seems to me that it will be faster”.

YAGNI principle – the programmer should not add functionality until deemed necessary.
Do it in the simplest way (ordinary pagination of one page), measure how it works on production, if it is slow, then try a different method, if the speed is satisfactory, leave it as it is.

From my own practice – an application that retrieves data from a table containing about 80,000 records, the main table is joined with 4-5 additional lookup tables, the whole query is paginated, about 25-30 records per page, about 2500-3000 pages in total. Database is Oracle 12c, there are indexes on a few columns, queries are generated by Hibernate. Measurements on production system at the server side show that an average time (median – 50% percentile) of retrieving one page is about 300 ms. 95% percentile is less than 800 ms – this means that 95% of requests for retrieving a single page is less that 800ms, when we add a transfer time from the server to the user and a rendering time of about 0.5-1 seconds, the total time is less than 2 seconds. That’s enough, users are happy.

And some theory – see this answer to know what is purpose of Pagination pattern

What are some ways we can use machine learning and artificial intelligence for algorithmic trading in the stock market?
What are some good datasets for Data Science and Machine Learning?
Machine Learning Engineer Interview Questions and Answers

  • Database Administrator Roadmap
    by Mohammad Usama (Database on Medium) on October 7, 2024 at 3:18 am

    The role of a Database Administrator (DBA) is critical in today’s data-driven world. DBAs are responsible for the performance, security…Continue reading on Stackademic »

  • Is the main (or only) advantage of Graph Databases the query language?
    by /u/Vizdun (Database) on October 7, 2024 at 12:19 am

    I'm making this pet database thingy and I started off with the standard graph setup since that seemed most useful at the time: homogeneous nodes, dynamic properties, named edges (though with a ban on one-to-many relations since that makes for nicer things down the road), etc. Overtime though, cracks began to show: i started giving every node a "type" property because it's very useful to know what the node is actually supposed to be, so that you can for example query name = "Redis" and be sure the result is the database and not the company schemaless is kind of memory inefficient and also kind of painful to work with (?) so you could have (optional) schemas for nodes related to the type edges are really snowflake values, they're like any other value until they're not and then you gotta make a bunch of special cases for them, and at the end of the day, you still gotta query nodes by properties, so you might as well just do value based joins with some syntax sugar, those would be of course a bit less performant but i'm sure that could be optimized so what i'm left with is pretty much (optionally schemaless) tables with some edge-like syntax sugar, did i lose anything i missed by this? submitted by /u/Vizdun [link] [comments]

  • SQL 101 | Chapter 1: Understanding SQL and Databases
    by Yujin (Database on Medium) on October 7, 2024 at 12:11 am

    How do databases serve as efficient repositories of structured data, and how does SQL (Structured Query Language) enable businesses to…Continue reading on Medium »

  • Oracle Exadata: A Developer’s Guide to Boosting Database Performance and Scalability
    by InterviewBuddies (Database on Medium) on October 6, 2024 at 11:00 pm

    As a developer, ensuring your applications perform well even as data scales up is crucial. Oracle Exadata is a purpose-built platform that…Continue reading on Medium »

  • Unlock Spark in 8 Key Areas: Master Apache’s Powerhouse Engine
    by Jackson Williams (Database on Medium) on October 6, 2024 at 9:13 pm

    Continue reading on Medium »

  • I want to make a Chess database for Chessable.
    by /u/Away-Following-3236 (Database) on October 6, 2024 at 9:02 pm

    How do I combine all of my PGN/FEN codes into one .db3 or .csv file. Is there a proper way to format this or what? Tysm for any help EDIT: I meant to say En Croissant in the title, not chessable, sorry. submitted by /u/Away-Following-3236 [link] [comments]

  • How to install mongodb/laravel-mongodb using the Ubuntu terminal in Windows 11.
    by Mohamed ABNOUNE (Database on Medium) on October 6, 2024 at 8:28 pm

    Introduction:Continue reading on Medium »

  • Últimos Jogadores Atualizados — 06/10/2024
    by Tudo pelo Futebol (Database on Medium) on October 6, 2024 at 8:14 pm

    Continue reading on Medium »

  • Últimos Jugadores Actualizados — 06/10/2024
    by Todo por el Fútbol (Database on Medium) on October 6, 2024 at 8:14 pm

    Continue reading on Medium »

  • Last Updated Players — 10/06/2024
    by Everything for Football (Database on Medium) on October 6, 2024 at 8:14 pm

    Continue reading on Medium »

  • How to Deploy Nyrkiö Locally Using Docker
    by Wagner Franchin (Database on Medium) on October 6, 2024 at 7:12 pm

    A Step-by-Step Guide to Help You Get the Application Up and RunningContinue reading on ITNEXT »

  • SQL for Beginners | Day 7
    by Jayabanerjee (Database on Medium) on October 6, 2024 at 6:28 pm

    SQL ClausesContinue reading on Medium »

  • How to send mail from PostgreSQL database using Notify/Listen
    by /u/riddinck (Database) on October 6, 2024 at 12:48 pm

    submitted by /u/riddinck [link] [comments]

  • Please give advice on my database diagram
    by /u/Basic-Advertising446 (Database) on October 6, 2024 at 5:27 am

    This is my first time designing/ diagram of a database. I have yet to put datatypes because i thought they were self-explanatory but please advice me in case of any relationship problems between the table and anything i did wrong or redudant. Looking forward to the help. Thank You https://preview.redd.it/8v72qssbn2td1.png?width=1222&format=png&auto=webp&s=55fb89677be7d1bc60d1541da9f7a035d4b0e8e3 submitted by /u/Basic-Advertising446 [link] [comments]

  • Storing static data in time series database?
    by /u/WeezeAlarm (Database) on October 6, 2024 at 5:16 am

    I am wondering if you should store static data that doesn't change, like for example stock names or sector. I need to do this so I can filter/sort by time series data as well as static data. submitted by /u/WeezeAlarm [link] [comments]

  • Is data normalization needed in 2024? How much normalization is actually required and when should it be done?
    by /u/Notalabel_4566 (Database) on October 5, 2024 at 8:13 pm

    Can you provide some examples? submitted by /u/Notalabel_4566 [link] [comments]

  • Relational algebra
    by /u/lewishamilton4413 (Database) on October 5, 2024 at 3:17 pm

    Hello guys I'm studying database systems as a student. I wanted to know what purpose does relational algebra and calculus serve in databases thank you. submitted by /u/lewishamilton4413 [link] [comments]

  • Is this a good ERD model?
    by /u/dizson (Database) on October 5, 2024 at 9:23 am

    submitted by /u/dizson [link] [comments]

  • Trying to understand main components to a DB.
    by /u/masterofnone811 (Database) on October 5, 2024 at 2:32 am

    Tasked with building out my companies CMDB/Asset management program. There basically is nothing but some spreadsheets different teams have been tracking things in. I have been good with all the other equipment/software up to this point but documenting the DB’s has been a struggle. Look for suggestions for, high level, what are the main areas to track as part of DB in a CMDB. This build out will be relational, attribute references to applications and servers the DB work with are already planned. Not looking to recreate the entire schema in the CMDB. Want the main components and the important details for them. Diagrams/image suggestions are welcome. submitted by /u/masterofnone811 [link] [comments]

  • Would MongoDB Be Scalable Choice for a Chat App?
    by /u/travistrue (Database) on October 4, 2024 at 7:53 pm

    I’ve wanted to build an app that has a chat component as part of it. Users can just send plain text as an MVP, but I’d eventually want to allow users to embed things such as web links, photos, videos into their messages. Honestly, when they upload photos and videos, they’d get uploaded to an AWS S3 bucket, and then the database would just embed a hyperlink to that thing. In the end, each “message” would be a block of text. Each message would be associated to a “conversation”. Multiple “users” would be associated to a conversation. Now, if I went the relational approach, I see a many-to-many relationship between a “users” table and a “messages” table where the cross (join) table would be the “conversations” table. That’s simple, but would a non-relational database (like MongoDB) be better suited for this? My concern with relational databases is that messages can accrue very, VERY quickly across many different conversations. Especially if the same user is a part of several conversations… What if the app had (theoretically) millions of new messages every single day? That one table gets massive quickly. We can’t shard things much either. A tenant-based database approach could help, but I don’t really have a use-case for tenants in this case. What if I used a relational database to keep track of the list of users and conversations (the heavily relational side), but then stored the contents of each conversation in a MongoDB collection? Each time a new conversation is created, I’d create a new Conversation record in my relational DB, and then create a new MongoDB collection that’s named after the new conversation’s ID. This way, I don’t have to store all messages for every conversation on the same spot. I can store all messages them by conversation (MongoDB collection). I can come up with ways of sharding collections too. The nice thing is that all the relational stuff is kept completely in relational database which I can leverage transactions with. Heck, I can even wrap my MongoDB call into my SQL transaction cuz it’s at the end. If MongoDB fails, then that one mutable operation doesn’t happen anyway, and I can roll back the relational part of that whole query too. Thoughts? submitted by /u/travistrue [link] [comments]

  • "File is not a database" - .db file that I'm having trouble accessing
    by /u/Gronendael (Database) on October 4, 2024 at 2:41 pm

    My company switched our business software a couple years ago. We sometimes still need to access the old software, but it is now being discontinued and going away. They sent us a copy of our data with 2 files, a .db and a .log. According to the company, the files are in a "MySQL format". I worked in software and light programming 20 years ago including with SQL Server, SQL statements, Access, MySQL, and a couple others I can't remember...so I'm not totally ignorant about this sort of thing and I can generally figure out with some Google searches what I need to do to access something like this. I've tried DBeaver, SQLLiteStudio, MySQL Server/Workbench, and DB Browser and cannot access it. Get this "File is not a database" error in most of them. Tried changing the file extension to .sql but that didn't work either. The company will not help me and just told me that that's all they can do and that this is the same format they send to other companies that integrated with their software and there was never a problem. Any ideas what I'm missing or what I might try to access this file? My goal is to access the data and create a simple interface so we can view some of the information in it like client records and history. submitted by /u/Gronendael [link] [comments]

  • The Hell of Documenting an SQL database?
    by /u/gxslash (Database) on October 3, 2024 at 11:05 am

    I wonder how could I professionally and efficiently document a database. I have a bunch of postgreSQL databases. I would like to document them and search for the different methods people use. I came with this question on stackoverflow. And there are two questions appeared in my mind: 1- Is there really a specification for database documenting? Any specified formatting, method, rule, etc? 2- Why there is so much tools while you can easily comment your tables & fields inside postgreSQL? Sure, if you have multiple different DBMs (postgreSQL, msSQL, mongo, Cassandra ...) and would like to document them in a single, it is better to stick with single documentation method. I don't think most startups use multiple DBMs, but in the link above, there is only single person suggesting commenting. submitted by /u/gxslash [link] [comments]

  • Can my relationship have a key with same name as a key in another entity? Talking about ServiceID
    by /u/6null9 (Database) on October 2, 2024 at 12:07 pm

    submitted by /u/6null9 [link] [comments]

  • Please suggest a relational database with a Javascript API that doesn't rely on SQL
    by /u/BjornMoren (Database) on October 1, 2024 at 7:50 am

    I am currently using PostgreSQL but have earlier used MSSQL and MySQL for many years. I'm dead tired of SQL as a language. Sure, very convenient for low and medium complexity queries, but a nightmare for highly complex queries and very hard to debug due to its declarative nature. You never know exactly what happens in the execution. But I like relational databases (schemas, indexes, constraints and foreign keys). They map very well to how I think about data in general. So I hope to avoid working with key-value stores, document databases, or object databases. So I'm thinking that someone is probably as fed up as me and has written an extension to PostgreSQL where you bypass SQL entirely. But I haven't found any. I want a Javascript API similar to the one MongoDB uses. But one that doesn't get translated to SQL behind the scenes, because that will set a serious limitation on how flexible that API can be. A Javascript API that talks directly to the low level libraries of PostgreSQL. I could switch to MongoDB I guess. It is well known and robust. I like the API. But it is a document database with BSON/JSON entries, which means a lot of redundant data and lower performance even when you use schemas and carefully constructed indexes. I might accept that. Do you have any suggestions? Robust database, high performance, can handle large datasets, for a backend server Has a Javascript query API that does not resemble SQL in the slightest, not even reliant on SQL, where I can put the Javascript on the server itself (stored procedure) and set breakpoints. I found Realm from MongoDB which looks exactly like what I want. But it is designed for mobile, so I'm weary to take a chance with on a server backend. submitted by /u/BjornMoren [link] [comments]

  • Help with a Petri Net System
    by /u/Mlafe (Database) on September 30, 2024 at 10:25 pm

    Hi all, I’m trying to write a Petri net system, part of which needs to check if something is rented out or not for availability. I don’t know if this is the subreddit for it, but the one for Petri is completely dead, so I’m gonna ask here. submitted by /u/Mlafe [link] [comments]

  • Database for Grocery Stores in USA?
    by /u/Intrepid-Way-1756 (Database) on September 30, 2024 at 2:57 pm

    Is there a database that's available online (free or paid) that has a list of all items sold on Walmart, Kroger, Target, etc's online websites? Ideally a breakdown of all SKUs with categories and sub-categories listed. submitted by /u/Intrepid-Way-1756 [link] [comments]

  • Getting started with Skyward?
    by /u/DreamcastSonic (Database) on September 30, 2024 at 11:23 am

    New to database work here. Using a program called Skyward and am kinda left to myself to figure it out. The modules on its site from what I've seen are quite unhelpful in terms of what I actually need to know since there's so much going on; what are some essential paths in the program or even just some basic things I should know? submitted by /u/DreamcastSonic [link] [comments]

  • Where to start?
    by /u/ComfortableSelect137 (Database) on September 30, 2024 at 8:42 am

    https://preview.redd.it/wqjgwufyswrd1.png?width=859&format=png&auto=webp&s=7a08a4becda2b035ec8d98e7490860b09c66cf16 Hi everyone, may you kindly assist. I am 3rd year Computer Science Student (Bachelor's) and one of my final modules is titled Database Fundamentals. The book in the picture is one of the resources that we are using, I have never done anything with Database related. I have been looking for free courses on YouTube but I feel like I am not finding the right ones (I watched an hour of this one https://www.youtube.com/watch?v=4cWkVbC2bNE&t=1889s ), for finals I am required to submit a Database related Project (MySQL). May you kindly recommend the right courses to watch that will help me understand better, also where I can learn SQL. Thank you. submitted by /u/ComfortableSelect137 [link] [comments]

  • ERD Relationship Attributes
    by /u/Eastern-Job-8028 (Database) on September 30, 2024 at 3:45 am

    I am trying to figure out a way or line of thinking to determine if an attribute should belong to an entity, or rather the relationship that connects an entity to another. An example: Customer (entity) places (relationship) order (entity). Would such entities of (specified quantity of an item, date of receipt, expected ship date, actual ship date, price) be attributes of the relationship (places) or belong to the order entity? submitted by /u/Eastern-Job-8028 [link] [comments]

  • Competing for the JOB with a Triplestore
    by /u/Historical_Bat_9793 (Database) on September 29, 2024 at 8:55 pm

    submitted by /u/Historical_Bat_9793 [link] [comments]

  • My latest article on Medium - Scaling ClickHouse: Achieve Faster Queries using Distributed Tables
    by /u/SAsad01 (Database) on September 29, 2024 at 1:07 pm

    I am sharing my latest Medium article that covers Distributed table engine and distributed tables in ClickHouse. It covers creation of distributed tables, data insertion, and query performance comparison. Read here: https://medium.com/@suffyan.asad1/scaling-clickhouse-achieve-faster-queries-using-distributed-tables-1c966d98953b ClickHouse is a fast, horizontally scalable data warehouse system, which has become popular due to its performance and ability to handle big data. submitted by /u/SAsad01 [link] [comments]

  • Can someone help me out with this ER diagram? I don't think it makes much sense, but I don't know the proper way to convert these table schemas into one.
    by /u/DapperNurd (Database) on September 29, 2024 at 3:03 am

    submitted by /u/DapperNurd [link] [comments]

  • Do I have to use both SQLite and MySQL for a local and online database or can I use one? Is it that much of a problem to use both? Trying to create Inventory management system and want access to the database both locally and online from the front end.
    by /u/_snapdowncity (Database) on September 28, 2024 at 11:18 am

    submitted by /u/_snapdowncity [link] [comments]

  • 18 months of pgvector learnings in 47 minutes (PostgreSQL)
    by /u/k4lki (Database) on September 27, 2024 at 9:57 pm

    submitted by /u/k4lki [link] [comments]

  • Why do I hear ribbit noises whenever my dad is working?
    by /u/Alyssarr9fox (Database) on September 26, 2024 at 4:20 pm

    hi i'd like to ask why do i hear ribbit noises whenever he's working.. i assume it's a database thing because my dad works in databases but when i search "database ribbit" up on google nothing works.. so i'd like to ask why is there ribbit noises? i can't ask him because when i do he just gives me an answer of like he's busy and databases and all that.. please someone help thank you submitted by /u/Alyssarr9fox [link] [comments]

Posted on

Budget to start a web app built on the MEAN stack

I want to start a web app built on the MEAN stack (mongoDB, express.js, angular, and node.js). How much would it cost me to host this site? What resources are there for hosting websites built on the MEAN stack?

I went through the same questions and concerns and I actually tried a couple of different cloud providers for similar environments and machines.

Web Apps Feed

  1. At Digital Ocean, you can get a fully loaded machine to develop and host at $5 per month (512 MB RAM, 20 GB disk ). You can even get a $10 credit by using this link of mine.[1] It is very easy to sign up and start. Just don’t use their web console to connect to your host. It is slow. I recommend using ssh client to connect and it is very fast.
  2. GoDaddy will charge you around 8$ per month for a similar MEAN stack host (512 MB RAM, 1 core processor, 20 Gb disk ) for your MEAN Stack development.
  3. Azure use bitmani’s mean stack on minimum DS1_V2 machine (1core, 3.5 gB RAM) and your average cost will be $52 per month if you never shut down the machine. The set up is a little bit more complicated that Digital Ocean, but very doable. I also recommend ssh to connect to the server and develop.
  4. AWS also offers Bitmani’s MEAN stack on EC2 instances similar to Azure DS1V2 described above and it is around $55 per month.
  5. Other suggestions

All those solutions will work fine and it all depends on your budget. If you are cheap like me and don’t have a big budget, go with Digital Ocean and start with $10 off with this code.

Posted on

Basic Gotcha Linux Questions for IT DevOps and SysAdmin Interviews

Some IT DevOps, SysAdmin, Developer positions require the knowledge of basic linux Operating System. Most of the time, we know the answer but forget them when we don’t practice very often. This refresher will help you prepare for the linux portion of your IT interview by answering some gotcha Linux Questions for IT DevOps and SysAdmin Interviews.

Get a $10 credit to have your own linux server for your MEAN STACK development and more. It is only $5 per month for a fully loaded Ubuntu machine.

Latest Linux Feeds

I- Networking:

  1. How many bytes are there in a MAC address?
    48.
    MAC, Media Access Control, address is a globally unique identifier assigned to network devices, and therefore it is often referred to as hardware or physical address. MAC addresses are 6-byte (48-bits) in length, and are written in MM:MM:MM:SS:SS:SS format.
  2. What are the different parts of a TCP packet?
    The term TCP packet appears in both informal and formal usage, whereas in more precise terminology segment refers to the TCP protocol data unit (PDU), datagram to the IP PDU, and frame to the data link layer PDU: … A TCP segment consists of a segment header and a data section.
  3. Networking: Which command is used to initialize an interface, assign IP address, etc.
    ifconfig (interface configuration). The equivalent command for Dos is ipconfig.
    Other useful networking commands are: Ping, traceroute, netstat, dig, nslookup, route, lsof
  4. What’s the difference between TCP and UDP; Between DNS TCP and UDP?
    There are two types of Internet Protocol (IP) traffic. They are TCP or Transmission Control Protocol and UDP or User Datagram Protocol. TCP is connection oriented – once a connection is established, data can be sent bidirectional. UDP is a simpler, connectionless Internet protocol.
    The reality is that DNS queries can also use TCP port 53 if UDP port 53 is not accepted.
    DNS uses TCP for Zone Transfer over port :53.
    DNS uses UDP for DNS Queries over port :53.

  5. What are defaults ports used by http, telnet, ftp, smtp, dns, , snmp, squid?
    All those services are part of the Application level of the TCP/IP protocol.
    http => 80
    telnet => 23
    ftp => 20 (data transfer), 21 (Connection established)
    smtp => 25
    dns => 53
    snmp => 161
    dhcp => 67 (server), 68 (Client)
    ssh => 22
    squid => 3128
  6. How many host available in a subnet (Class B and C Networks)
  7. How DNS works?
    When you enter a URL into your Web browser, your DNS server uses its resources to resolve the name into the IP address for the appropriate Web server.
  8. What is the difference between class A, class B and class C IP addresses?
    Class A Network (/ 8 Prefixes)
    This network is 8-bit network prefix. IP address range from 0.0.0.0 to 127.255.255.255
    Class B Networks (/16 Prefixes)
    This network is 16-bit network prefix. IP address range from 128.0.0.0 to 191.255.255.255Class C Networks (/24 Prefixes)
    This network is 24-bit network prefix.IP address range from 192.0.0.0 to 223.255.255.255
  9. Difference between ospf and bgp?
    The first reason is that BGP is more scalable than OSPF. , and this, normal igp like ospf cannot perform. Generally speaking OSPF and BGP are routing protocols for two different things. OSPF is an IGP (Interior Gateway Protocol) and is used internally within a companies network to provide routing.

II- Operating System
1&1 Web Hosting

  1. How to find the Operating System version?
    $uname -a
    To check the distribution for redhat for example: $cat /etc/redhat –release
  2. How to list all the process running?
    top
    To list java processes, ps -ef | grep java
    To list processes on a specific port:
    netstat -aon | findstr :port_number
    lsof -i:80
  3. How to check disk space?
    df shows the amount of disk space used and available.
    du displays the amount of disk used by the specified files and for each subdirectories.
    To drill down and find out which file is filling up a drive: du -ks /drive_name/* | sort -nr | head
  4. How to check memory usage?
    free or cat /proc/meminfo
  5. What is the load average?
    It is the average sum of the number of process waiting in the queue and the number of process currently executing over the period of 1, 5 and 15 minutes. Use top to find the load average.
  6. What is a load balancer?
    A load balancer is a device that acts as a reverse proxy and distributes network or application traffic across a number of servers. Load balancers are used to increase capacity (concurrent users) and reliability of applications.
  7. What is the Linux Kernel?
    The Linux Kernel is a low-level systems software whose main role is to manage hardware resources for the user. It is also used to provide an interface for user-level interaction.
  8. What is the default kill signal?
    There are many different signals that can be sent (see signal for a full list), although the signals in which users are generally most interested are SIGTERM (“terminate”) and SIGKILL (“kill”). The default signal sent is SIGTERM.
    kill 1234
    kill -s TERM 1234
    kill -TERM 1234
    kill -15 1234
  9. Describe Linux boot process
    BIOS => MBR => GRUB => KERNEL => INIT => RUN LEVEL
    As power comes up, the BIOS (Basic Input/Output System) is given control and executes MBR (Master Boot Record). The MBR executes GRUB (Grand Unified Boot Loader). GRUB executes Kernel. Kernel executes /sbin/init. Init executes run level programs. Run level programs are executed from /etc/rc.d/rc*.d
    Mac OS X Boot Process:

    Boot ROMFirmware. Part of Hardware system
    BootROM firmware is activated
    POSTPower-On Self Test
    initializes some hardware interfaces and verifies that sufficient memory is available and in a good state.
    EFI Extensible Firmware Interface
    EFI does basic hardware initialization and selects which operating system to use.
    BOOTX boot.efi boot loader
    load the kernel environment
    Rooting/Kernel The init routine of the kernel is executed
    boot loader starts the kernel’s initialization procedure
    Various Mach/BSD data structures are initialized by the kernel.
    The I/O Kit is initialized.
    The kernel starts /sbin/mach_init
    Run Level mach_init starts /sbin/init
    init determines the runlevel, and runs /etc/rc.boot, which sets up the machine enough to run single-user.
    rc.boot figures out the type of boot (Multi-User, Safe, CD-ROM, Network etc.)
  10. List services enabled at a particular run level
    chkconfig –list | grep 5:0n
    Enable|Disable a service at a specific run level: chkconfig on|off –level 5
  11. How do you stop a bash fork bomb?
    Create a fork bomb by editing limits.conf:
    root hard nproc 512
    Drop a fork bomb as below:
    :(){ :|:& };:
    Assuming you have access to shell:
    kill -STOP
    killall -STOP -u user1
    killall -KILL -u user1
  12. What is a fork?
    fork is an operation whereby a process creates a copy of itself. It is usually a system call, implemented in the kernel. Fork is the primary (and historically, only) method of process creation on Unix-like operating systems.
  13. What is the D state?
    D state code means that process is in uninterruptible sleep, and that may mean different things but it is usually I/O.

III- File System

  1. What is umask?
    umask is “User File Creation Mask”, which determines the settings of a mask that controls which file permissions are set for files and directories when they are created.
  2. What is the role of the swap space?
    A swap space is a certain amount of space used by Linux to temporarily hold some programs that are running concurrently. This happens when RAM does not have enough memory to hold all programs that are executing.
  • What is the role of the swap space?
    A swap space is a certain amount of space used by Linux to temporarily hold some programs that are running concurrently. This happens when RAM does not have enough memory to hold all programs that are executing.
  • What is the null device in Linux?
    The null device is typically used for disposing of unwanted output streams of a process, or as a convenient empty file for input streams. This is usually done by redirection. The /dev/null device is a special file, not a directory, so one cannot move a whole file or directory into it with the Unix mv command.You might receive the “Bad file descriptor” error message if /dev/null has been deleted or overwritten. You can infer this cause when file system is reported as read-only at the time of booting through error messages, such as“/dev/null: Read-only filesystem” and “dup2: bad file descriptor”.
    In Unix and related computer operating systems, a file descriptor (FD, less frequently fildes) is an abstract indicator (handle) used to access a file or other input/output resource, such as a pipe or network socket.
  • What is a inode?
    The inode is a data structure in a Unix-style file system that describes a filesystem object such as a file or a directory. Each inode stores the attributes and disk block location(s) of the object’s data.

IV- Databases

  1. What is the difference between a document store and a relational database?
    In a relational database system you must define a schema before adding records to a database. The schema is the structure described in a formal language supported by the database and provides a blueprint for the tables in a database and the relationships between tables of data. Within a table, you need to define constraints in terms of rows and named columns as well as the type of data that can be stored in each column.In contrast, a document-oriented database contains documents, which are records that describe the data in the document, as well as the actual data. Documents can be as complex as you choose; you can use nested data to provide additional sub-categories of information about your object. You can also use one or more document to represent a real-world object.
  2. How to optimise a slow DB?
    • Rewrite the queries
    • Change indexing strategy
    • Change schema
    • Use an external cache
    • Server tuning and beyond
  3. How would you build a 1 Petabyte storage with commodity hardware?
    Using JBODs with large capacity disks with Linux in a distributed storage system stacking nodes until 1PB is reached.
    JBOD (which stands for “just a bunch of disks”) generally refers to a collection of hard disks that have not been configured to act as a redundant array of independent disks (RAID) array.
    JBOD

V- Scripting

  1. What is @INC in Perl?
    The @INC Array. @INC is a special Perl variable that is the equivalent to the shell’s PATH variable. Whereas PATH contains a list of directories to search for executables, @INC contains a list of directories from which Perl modules and libraries can be loaded.
  2. Strings comparison – operator – for loop – if statement
  3. Sort access log file by http Response Codes
    Via Shell using linux commands
    cat sample_log.log | cut -d ‘”‘ -f3 | cut -d ‘ ‘ -f2 | sort | uniq -c | sort -rn
  4. Sort access log file by http Response Codes Using awk
    awk ‘{print $9}’ sample_log.log | sort | uniq -c | sort -rn
  5. Find broken links from access log file
    awk ‘($9 ~ /404/)’ sample_log.log | awk ‘{print $7}’ sample_log.log | sort | uniq -c | sort -rn
  6. Most requested page:
    awk -F\” ‘{print $2}’ sample_log.log | awk ‘{print $2}’ | sort | uniq -c | sort -r
  7. Count all occurrences of a word in a file
    grep -o “user” sample_log.log | wc -w

Learn more at http://career.guru99.com/top-50-linux-interview-questions/

Real Time Linux Jobs

Posted on

Install and run your first noSQL MongoDB on Mac OSX

Amazon SQL vs NoSQL

Install and run your first noSQL MongoDB on Mac OSX

Classified as a NoSQL database, MongoDB is an open source, document-oriented database designed with both scalability and developer agility in mind. Instead of storing your data in tables and rows as you would with a relational database, in MongoDB you store JSON-like documents with dynamic schemas; This makes the integration of data in certain types of application easier and faster.
Why?
MongoDB can help you make a difference to the business. Tens of thousands of organizations, from startups to the largest companies and government agencies, choose MongoDB because it lets them build applications that weren’t possible before. With MongoDB, these organizations move faster than they could with relational databases at one tenth of the cost. With MongoDB, you can do things you could never do before.

    1. Install Homebrew
      $ /usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)”
      Homebrew installs the stuff you need that Apple didn’t.
      $ brew install wget
    2. Install MongoDB
      $ brew install mongodb
    3. Run MongoDB
      Create the data directory: $ mkdir -p /data/db
      Set permissions for the data directory:$ chown -R you:yourgroup /data/db then chmod -R 775 /data/db
      Run MongoDB (as non root): $ mongod
    4. Begin using MongoDB.(MongoDB will be running as soon as you ran mongod above)Open another terminal and run: mongo

Install and run your first noSQL MongoDB on Mac OSX

References: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/


Ace the 2023 AWS Solutions Architect Associate SAA-C03 Exam with Confidence Pass the 2023 AWS Certified Machine Learning Specialty MLS-C01 Exam with Flying Colors

List of Freely available programming books - What is the single most influential book every Programmers should read



#BlackOwned #BlackEntrepreneurs #BlackBuniness #AWSCertified #AWSCloudPractitioner #AWSCertification #AWSCLFC02 #CloudComputing #AWSStudyGuide #AWSTraining #AWSCareer #AWSExamPrep #AWSCommunity #AWSEducation #AWSBasics #AWSCertified #AWSMachineLearning #AWSCertification #AWSSpecialty #MachineLearning #AWSStudyGuide #CloudComputing #DataScience #AWSCertified #AWSSolutionsArchitect #AWSArchitectAssociate #AWSCertification #AWSStudyGuide #CloudComputing #AWSArchitecture #AWSTraining #AWSCareer #AWSExamPrep #AWSCommunity #AWSEducation #AzureFundamentals #AZ900 #MicrosoftAzure #ITCertification #CertificationPrep #StudyMaterials #TechLearning #MicrosoftCertified #AzureCertification #TechBooks

Top 1000 Canada Quiz and trivia: CANADA CITIZENSHIP TEST- HISTORY - GEOGRAPHY - GOVERNMENT- CULTURE - PEOPLE - LANGUAGES - TRAVEL - WILDLIFE - HOCKEY - TOURISM - SCENERIES - ARTS - DATA VISUALIZATION
zCanadian Quiz and Trivia, Canadian History, Citizenship Test, Geography, Wildlife, Secenries, Banff, Tourism

Top 1000 Africa Quiz and trivia: HISTORY - GEOGRAPHY - WILDLIFE - CULTURE - PEOPLE - LANGUAGES - TRAVEL - TOURISM - SCENERIES - ARTS - DATA VISUALIZATION
Africa Quiz, Africa Trivia, Quiz, African History, Geography, Wildlife, Culture

Exploring the Pros and Cons of Visiting All Provinces and Territories in Canada.
Exploring the Pros and Cons of Visiting All Provinces and Territories in Canada

Exploring the Advantages and Disadvantages of Visiting All 50 States in the USA
Exploring the Advantages and Disadvantages of Visiting All 50 States in the USA


Health Health, a science-based community to discuss health news and the coronavirus (COVID-19) pandemic

Today I Learned (TIL) You learn something new every day; what did you learn today? Submit interesting and specific facts about something that you just found out here.

Reddit Science This community is a place to share and discuss new scientific research. Read about the latest advances in astronomy, biology, medicine, physics, social science, and more. Find and submit new publications and popular science coverage of current research.

Reddit Sports Sports News and Highlights from the NFL, NBA, NHL, MLB, MLS, and leagues around the world.

Turn your dream into reality with Google Workspace: It’s free for the first 14 days.
Get 20% off Google Google Workspace (Google Meet) Standard Plan with  the following codes:
Get 20% off Google Google Workspace (Google Meet) Standard Plan with  the following codes: 96DRHDRA9J7GTN6 96DRHDRA9J7GTN6
63F733CLLY7R7MM
63F7D7CPD9XXUVT
63FLKQHWV3AEEE6
63JGLWWK36CP7WM
63KKR9EULQRR7VE
63KNY4N7VHCUA9R
63LDXXFYU6VXDG9
63MGNRCKXURAYWC
63NGNDVVXJP4N99
63P4G3ELRPADKQU
With Google Workspace, Get custom email @yourcompany, Work from anywhere; Easily scale up or down
Google gives you the tools you need to run your business like a pro. Set up custom email, share files securely online, video chat from any device, and more.
Google Workspace provides a platform, a common ground, for all our internal teams and operations to collaboratively support our primary business goal, which is to deliver quality information to our readers quickly.
 Get 20% off Google Workspace (Google Meet) Business Plan (AMERICAS): M9HNXHX3WC9H7YE
C37HCAQRVR7JTFK
C3AE76E7WATCTL9
C3C3RGUF9VW6LXE
C3D9LD4L736CALC
C3EQXV674DQ6PXP
C3G9M3JEHXM3XC7
C3GGR3H4TRHUD7L
C3LVUVC3LHKUEQK
C3PVGM4CHHPMWLE
C3QHQ763LWGTW4C
Even if you’re small, you want people to see you as a professional business. If you’re still growing, you need the building blocks to get you where you want to be. I’ve learned so much about business through Google Workspace—I can’t imagine working without it. (Email us for more codes)