Uncategorized

Lab: Running A Java Data flow Program In this lecture, you’ll understand why you would choose to use the Pair Do class while running transforms in a Java pipeline, we’ll implement the same Grep program that we saw earlier, but this time we are going to do it in Java and we’ll run it on the Cloud and see how that works. The code for this Java implementation is also part of Google’s Code Labs for the Data Engineer certification. This code is available in training data analyst courses. Data…

Data Flow Intro Next up, let’s talk about data flow. Data flow is a way in which to carry out transformations on your data. This is loosely similar to some amalgam of Pig and Spark in the Hadoop ecosystem world. As we discuss data flow, here is a question that I’d like you to keep in mind and try and ponder over. It’s a question we’ve seen before while talking about Pig. In a typical ETL use case, is hive or BigQuery going to be a source or a sync?…

Column Families Here’s a question, a rather open ended question, which I’d like you to keep in mind as you watch this video. How does the choice of a row key affect physical storage in Big table or in HBase? The manner in which data is laid out in a traditional database is very different from the layout in a columnar store. And that layout has implications, important implications, for the performance of HBase and Bitable. So let’s understand these differences in a little more detail. A traditional database has…

Lab: Subqueries And Joins At the end of this lecture, you’ll be familiar with concatenating strings together in a query in order to format your data the way you want to. This lecture is a continuation of the lab that we worked on earlier. We’ll continue to work on aggregate and boolean functions in BigQuery. In the last lecture, we left off where we’d run a query to calculate the total number of flights and the number of flights that were delayed per airline from La Guardia for a particular…

Lab: Loading JSON Data With Nested Tables At the end of this lecture, try and answer this question how would you access nested JSON fields when data with this kind of nested JSON is loaded into BigQuery? In this lecture, we’ll see how we can load nested JSON data into BigQuery with or without specifying a schema. We’ll start off in the BigQuery web console. Let’s create a brand new data set within our test project. We’ll call the data set person data. We located in the US. Click on…

BigQuery Intro As we begin our discussion of BigQuery, which is a Google Cloud Platform’s workhorse data warehousing and business intelligence and analytics processing service, here is a question that I’d like you to keep in mind. BigQuery takes Hive’s schema on Read to the next level. How does it do that? What exactly does BigQuery do as far as schema enforcement goes? Recall here for context that relational database management systems have schema on right? They enforce schema constraints each time you attempt to write to the database, and…

The 6 dB Rule Now there is this idea of the six-DB rule. They call it the inverse square law. So let’s think about it. Let me put my antenna at the top this time. Let’s think about the power. We know that we have our best power where the antenna begins, and that power is going to go down as we move away from the antenna. So basically, the inverse square law says that if I double the distance from the source to wherever I am, my total DBMS…

Watt Part1 So when we start off with the measurement of a watt, and remember, we’re going to be talking about milliwatts, or one thousandth of a watt, it is a basic unit of power. And some of the ideas here are to say, you know, if I had this hose, this water hose, that was going to be used for whatever reason. And when we talk about the water that’s flowing through the hose, we would call that the current that we’re sending. What we’ll do is talk about…

Semidirectional Antenna Now, a semidirectional antenna is trying to direct that signal in a specific direction. You might remember that I gave the example of this light pole, right, which sticks out of the ground and then has a piece that hangs over and then has the light that shines down and covers a pretty good area. but it’s certainly not omnidirectional. It’s just trying to light up an area, like I said, of a parking lot or a road or something else, maybe the front of your house, to…

Components and Measurements of RF In this module, we’re going to make you mathematicians, whereas in other parts of the course, we’ll make you physicists. So I hope that you have your mathematics hat on, and we’re going to talk about the components of RF communications, and we’re also going to look at things like not only the components but how they work together to help us get a certain type of strength of signal. And we’ll look at those power units and compare them. Then we’re going to get…