AWS Cloud Guide

In this blog I’m trying to cover the high level explanation of AWS Cloud Services.

Cloud Concept & Technology

It will cover AWS Core Services, Global Infrastructure & Design Principle.

AWS Cloud Computing models covers:

Infrastructure as a Service (IaaS)
Platform as a Service (PaaS)
Software as a Service (SaaS)

Region: Physical location. Cluster of data centers. Each Region consists of multiple, isolated, and physically separate AZ (Availability Zone). Each region comprises of at least 2 AZs. Region Example: US Ease (Ohio) Region, Asia Pacific (Mumbai) Region.

Availability Zone(AZ): One or more discrete data centers within region. AZ within a Region provide low-latency network connectivity to other AZ in the same Region. This helps in data replication.

Edge Locations: Used for caching the data for faster delivery with lower latency.

No of Edge Locations > AZ > Region

Elastic Compute Cloud (EC2): Provide scalable computing capacity in the Amazon cloud. Types of EC2 Instance :

On Demand – Recommended for short temp applications, irregular works that cannot be interrupted. No long term commitments, no upfront money, pay only for seconds instance are in running state.
Spot Instance – Unused EC2 available for less than on-demand price. Cost effective choice when you have no issues with interruption. Hourly price for spot instance is called spot price, which depends on the demand and supply trend of spare EC2. Instance will be interrupted, if the spot price exceeds the maximum price for your request or if capacity is no longer available.
Reserved Instance (RI) – Not physical instances but rather a billing discount applied to use of on-demand instance. Recommended when you have commitment to use EC2 over a 1 or 3 year term. Pricing is determined by Instance type, region, tenancy & platform.
Dedicated Instance – Physically isolated at the hardware level. It runs in VPC.

Dedicated Hosts is recommended for BYOL.

Reserved Instances Types:

Standard – Steady State usage. You cannot modify the instance i.e. convert.
Convertible – You can modify attributes of the RI as long as the exchange results in the creation of Reserved Instances of equal or greater value. Attributes like instance family, operating system, payment option & tenancy.
Scheduled – Scheduled RIs are available to launch within the time windows you reserve. This option allows you to match your capacity reservation to a predictable recurring schedule that only requires a fraction of a day, a week, or a month. You cannot modify the instance

Relational Database Service (RDS): Service that makes easy to setup, operate & scale a relational database in the Amazon cloud. Cost efficient & re-sizable features available. Refer AWS RDS Instance Setup blog for more details.

DynamoDB – NoSQL db service used for storing non-structured data.
Redshift – fast, fully managed data warehouse service that is specifically designed for Online Analytic Processing System (OLAP) i.e. Data-warehouse & Business Intelligence.
Aurora – Relational database engine that’s compatible with MySQL & PostgreSQL. Aurora is part of Amazon RDS service, it take care of tasks such as patching, backup & recovery.

Simple Storage Service (S3): – Secure, durable, highly-scalable object storage service. You can store any amount data from anywhere. It is designed for 99.999999999% (11 9’s) of durability. In single PUT function you can upload 1 file up-to max of 5TB. Types of S3:

Standard S3 – General Purpose for frequent accessed data.
S3 Standard – Infrequent Access (S3 Standard-IA) and S3 One Zone-Infrequent Access (S3 One Zone-IA) for long-lived, but less frequently accessed data.
S3 Intelligent – Tiering for data with changing access pattern or unpredictable access patterns.
S3 Glacier (S3 Glacier) and Amazon S3 Glacier Deep Archive (S3 Glacier Deep Archive) for archive.

S3 & Dynamo DB – AWS services is designed with native Multi-AZ

Elastic Load Balancing (ELB): Service that is used to distribute load is the AWS service.

ElastiCache: In-memory data store or cache. It improves the performance of applications by retrieve information from cache instead of disk/db. Redis or memcached.

Marketplace: Online store where one can sell/buy software that runs on AWS platform.

Database Migration Service (DMS) : Service helps user to migrate databases with minimal downtime. It can migrate the data between homogeneous (Oracle to Oracle) as well as heterogeneous system (Oracle to Amazon Aurora).

Simple Queue Service (SQS): Message queuing service that enables you to send, store, and receive messages b/w software applications, without losing messages.

Simple Notification Service (SNS) : Service that coordinates and manages the delivery or sending of messages to subscribing endpoints or clients. There are two types of clients – publishers and subscribers. Enables you to decouple micro-services, distributed systems, and server-less applications.

Simple Email Service (SES): Email messaging platform

CloudFront: Fast Content delivery network service which securely deliver content to global users like Image data or videos. It uses the global network of Edge Locations and Regional Edge Caches.

CloudWatch: Used to monitor the utilization of AWS resources and services. Basically you create metrics & retrieve the statistics based on those metrics. You can create alarms also using cloud watch.

Route 53: Global service that provides Domain Name System (DNS) services, domain name registration, and health-checking web services.

Elastic MapReduce (EMR) Big data processing and analysis.

Elastic File System (EFS): Shared file system for use with Cloud services and on-premises resources.

CloudFormation: Enable architects to manage Infrastructure as code.

Command Line Interface (CLI): Manage AWS services from the command line and automate them through scripts.

Software Development Kit (SDK): Allows you to interact with AWS services using your preferred programming language.

OpsWorks: Configuration management service that provides managed instances of Chef and Puppet. Chef and Puppet are automation platforms that allow you to use code to automate the configurations of your servers.

CloudTrail: Service that can be used to monitor all user interactions with the AWS environment. It is designed to log all actions performed which is usefull for governance, compliance & auditing.

Snowball: Petabyte-scale data transport solution into and out of the AWS Cloud.

Elastic Container Registry (ECR): Docker container registry that allows developers to store, manage, and deploy Docker container images.

Athena: is an interactive query service that is mainly used to analyze data in Amazon S3 using standard SQL.

CloudEndure: Provides disaster recovery and cloud migration to AWS from any physical, virtual, or cloud-based infrastructure.

AWS Lambda: is a compute service that lets you run code without provisioning or managing servers. Allow you to run applications without administration.

Security & Compliance

It will cover Shared Responsibility Model, Cloud Security, Compliance, Access Management & Support

Amazon Inspector: Automated security assessment service solution which assesses services for vulnerabilities or deviations from best practices. It helps improve the security and compliance of applications. It create a detailed report for all security findings which can be viewed using Amazon Inspector console.

AWS Config: Provides you configuration history, and change notifications to enable security and governance.

AWS Trusted Advisor: which provide you best practice checks and recommendations for cost optimization, security, fault tolerance, performance & service limits. It also provides security recommendations to protect your AWS environment.

AWS Shield: Distributed Denial of Service (DDoS) protection service that safeguards applications running on AWS.

AWS WAF: To control and absorb traffic, and deflect unwanted requests

AWS Support Plans: Developer, Business & Enterprise.

Technical Account Manager (TAM): Primary point of contact for ongoing support for enterprise level support . TAM work with AWS Solution Architects and provide technical expertise & best practice for all AWS services.

AWS Security team – Responsible for security services provided by AWS.
AWS Concierge – Responsible for billing and account management related issues.
AWS Abuse team – Can assist you for issues like Spam, denial of service attacks, port scanning, intrusion etc.

Infrastructure Event Management (IEM): Program available to enterprise level customers for event management support & planning. Example: Product launch, infra migration & marketing events etc.

Shared Controls & Responsibility: Apply to both the infrastructure layer and customer layers.

Patch Management
Configuration Management
Awareness & Training

Customer is responsible for protecting the data rest /transit for all services.

IAM Best Practice:

Least privilege – means granting users the required permissions to perform the tasks entrusted to them and nothing more.

Key Management Service (KMS): Create and control the encryption keys used to encrypt your data.

Artifacts: To manage agreements & compliance documents.

Personal Health Dashboard : AWS services health view. Alerts for event impacting AWS resources.

Service Health Dashboard : Information about the current status and availability of the AWS services any time.

Billing & Pricing

In AWS, most of the services are available with no upfront costs as it follows the pay-as-you-go pricing.

AWS Consolidated Billing: For billing purposes, this feature treats all the accounts in the organization as one account. You can combine the usage across all accounts in the organization to share the Reserved Instance discounts, volume pricing discounts, and Savings Plans.

AWS Cost & Usage Report: Accessing the detailed information available about your AWS costs and usage.

Total Cost of Ownership (TCO): Calculator allows customers to evaluate the savings from using AWS and compare an AWS Cloud environment to on-premises and co-location environments. Calculator matches your current infrastructure to the most cost-effective AWS offering.

AWS Budgets: gives you the ability to set custom budgets that alert you when your costs or usage exceed (or are forecasted to exceed) your budgeted amount.

Cost Explorer: Tool that you can use to view your costs and usage. You can forecast and also get recommendations on usage.

Thanks!

Happy Learning! Your feedback would be appreciated!
Follow @shobhitsinghIN

Oracle SQL – Scenario Questions – Part 2

In this blog we will explore some more latest Oracle SQL – Scenario questions!

Part 1 Blog Link: Oracle SQL Scenario Questions Part 1

Scenario 1. In below example remove the duplicate. Example: Records 1 & 2 are identical.

 
CREATE TABLE "DISTANCE"
  (
    "SOURCE_NAME" VARCHAR2(20 BYTE),
    "DEST_NAME"   VARCHAR2(20 BYTE),
    "DISTANCE"    NUMBER
  );

Insert into DISTANCE (SOURCE_NAME,DEST_NAME,DISTANCE) values ('DELHI','JAIPUR',281);
Insert into DISTANCE (SOURCE_NAME,DEST_NAME,DISTANCE) values ('JAIPUR','DELHI',281);
Insert into DISTANCE (SOURCE_NAME,DEST_NAME,DISTANCE) values ('DELHI','PUNE',1427);
Insert into DISTANCE (SOURCE_NAME,DEST_NAME,DISTANCE) values ('PUNE','DELHI',1427);
Insert into DISTANCE (SOURCE_NAME,DEST_NAME,DISTANCE) values ('DELHI','SPITI',731);

COMMIT;

Solution: You can use the LEAST/GREATEST function to find out the dupes.

SELECT d.*,
row_number() over (partition BY LEAST(SOURCE_NAME, DEST_NAME),
GREATEST(SOURCE_NAME, DEST_NAME), distance order by distance )AS RNUM
FROM DISTANCE d;

From this data set you can filter the rows using the RNUM column to fetch distinct rows.

WITH data_set AS
(SELECT d.*,
row_number() over (partition BY LEAST(SOURCE_NAME, DEST_NAME), GREATEST(SOURCE_NAME, DEST_NAME), distance order by distance ) AS RNUM
FROM DISTANCE d
)
SELECT * FROM data_set WHERE RNUM =1;

Scenario 2. In below example customer data is present for product with dates. Fetch the customer who came in all months of quarter at least once. Like Jan, Feb, Mar or Apr, May, June etc. Notice in below data-set C1 & C3 came in all months of quarter Q1 & Q2 respectively.

Solution: You can use the date functions to find out the months & quarter.

SELECT CUST_ID,
listagg (TO_CHAR(BILL_DATE,'MM'),'||') within GROUP (
ORDER BY BILL_DATE) months_name,
TO_CHAR(BILL_DATE,'q')
FROM SALES
GROUP BY CUST_ID,
TO_CHAR(BILL_DATE,'YYYY'),
TO_CHAR(BILL_DATE,'q');

WITH dataset AS
  (SELECT CUST_ID,
    listagg (TO_CHAR(BILL_DATE,'MM'),'||') within GROUP (
  ORDER BY BILL_DATE) months_name,
    TO_CHAR(BILL_DATE,'YYYY') year_,
    TO_CHAR(BILL_DATE,'q') quarter_
  FROM SALES
  GROUP BY CUST_ID,
    TO_CHAR(BILL_DATE,'YYYY'),
    TO_CHAR(BILL_DATE,'q')
  )
SELECT *
FROM dataset
WHERE months_name IN ('01||02||03','04||05||06','07||02||09','10||11||12') ;

Scenario 3. Output of this SQL: Select count(1), SUM(1) from dual where 1=2;

Select count(1), SUM(1) from dual where 1=2;

Scenario 4. Print 1st day of Month, Last Day, Mid, Week, Quarter for given date.

WITH dataset AS
  (SELECT to_date('10-JAN-2019','dd-mon-yyyy') date_ FROM dual
  union
  SELECT to_date('20-FEB-2019','dd-mon-yyyy') date_ FROM dual
  union
  SELECT to_date('05-JUN-2019','dd-mon-yyyy') date_ FROM dual
  )
SELECT to_char(date_,'DD-MM-YYYY'), to_char(date_,'Q') Quarter ,to_char(date_,'W') week,
trunc((date_),'MM') FIRST_DATE_OF_MONTH, last_day(date_) LAST_DATE_OF_MONTH
FROM dataset;

Scenario 5. In below example you have to print date twice. See sample date.

Input: Output Required:

Solution:

WITH data_set AS
  ( SELECT id_ FROM dataset
  UNION ALL
  SELECT id_ FROM dataset
  )
SELECT * FROM data_set ORDER BY 1;

Scenario 6. In below example you have ~ tilde separated date in column VAL. You have to break the string using SQL like below. Refer required output screenshot.

Input: Required Output:

CREATE TABLE DATASET ( ID_ NUMBER, VAL VARCHAR2(100));

INSERT INTO  DATASET VALUES (1, 'A~B~C~D');
INSERT INTO  DATASET VALUES (2, 'X~Y~Z');
INSERT INTO  DATASET VALUES (5, 'K');
COMMIT;

Solution:

WITH data_set AS
  ( SELECT id_,val, regexp_count(val,'~') sep_cnt FROM dataset
  ),
  numseries AS
  (SELECT rownum rnum FROM DUAL CONNECT BY rownum <= 10
  )
SELECT id_,
  regexp_substr(val,'[^~]+',1,numseries.rnum) VAL
FROM data_set
JOIN numseries
ON data_set.sep_cnt+1>=numseries.rnum
ORDER BY 1;

Scenario 7. Similar to Scenario 6, In below example you have data in VAL column & you have bring it in one row separated by double pipe ||.

Input : Required Output:

DROP TABLE DATASET;
CREATE TABLE DATASET ( ID_ NUMBER, VAL VARCHAR2(100));
INSERT INTO "DATASET" (ID_, VAL) VALUES ('1', 'A');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('1', 'B');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('1', 'C');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('1', 'D');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('2', 'Y');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('2', 'Z');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('2', 'X');
INSERT INTO "DATASET" (ID_, VAL) VALUES ('5', 'K');
COMMIT;

Solution:

SELECT ID_,
  listagg (VAL, '||') within GROUP (
ORDER BY VAL) NEW_VAL
FROM dataset
GROUP BY ID_;

Scenario 8. Rotating rows to column, see below screenshot.

Input: Output:

Solution : Using Pivot Clause

SELECT *
FROM
  (SELECT prod_id, month_, sales FROM sales_data
  ) pivot( MAX(sales) FOR month_ IN ('Jan' AS JAN, 'Feb' AS FEB, 'Mar' AS MAR, 'Apr' AS APR) );

Thanks!

Happy Learning! Your feedback would be appreciated!

Follow @shobhitsinghIN

Oracle Interval Partitioning

In the previous blog Oracle Range Partition we have explored how we can create range partition in oracle using date as partition key. Also we have seen one example where the range partition key is defined maximum upto ’01-APR-2019′ (Non-inclusive). But we are trying to insert ’20-OCT-2019′ date data, which is not getting mapped to any partition, that’s why ORA-14400 error occurred. See below error report provided by oracle.

SQL Error: ORA-14400: inserted partition key does not map to any partition
14400. 00000 – “inserted partition key does not map to any partition”

Here comes the Interval Partition, which is extension of Oracle Range Partition in which, interval partitions are automatically created by the database when data is inserted into the partition.

Monthly Partition:

CREATE TABLE T_ACCOUNT
  (
    V_ACCOUNT_NUM  VARCHAR2(100 CHAR),
    V_ACCOUNT_NAME VARCHAR2(200 CHAR),
    ACC_DATE DATE
  )
  PARTITION BY RANGE
  (
    ACC_DATE
  )
  INTERVAL
  (
    NUMTOYMINTERVAL(1, 'MONTH')
  )
  (
    PARTITION P_DEC2018 VALUES LESS THAN (TO_DATE('01-JAN-2019','DD-MON-YYYY'))
  );

After that we have inserted the data of future months, i.e. SEP & OCT for which Partitions were not available. See below insert statements.

INSERT INTO "SAIL_IN_DEV"."T_ACCOUNT" (V_ACCOUNT_NUM, V_ACCOUNT_NAME, ACC_DATE) VALUES ('1234567890', 'ABC', TO_DATE('20-SEP-19', 'DD-MON-RR'));
INSERT INTO "SAIL_IN_DEV"."T_ACCOUNT" (V_ACCOUNT_NUM, V_ACCOUNT_NAME, ACC_DATE) VALUES ('1234567891', 'ABC', TO_DATE('20-OCT-19', 'DD-MON-RR'));
COMMIT;

After committing the data let see the automatic partitions created by Oracle :

Notice that a system generated partition named SYS_P601 & SYS_P402 has been created after inserting rows with a partition key greater than the provided partition.

Fetching the data from particular partition : In below example you will able to see only match partition data.

SELECT * FROM T_ACCOUNT partition (SYS_P601);

It would be really painful to look up the system generated partition name every time, if you have to query directly partition. Therefore, oracle provided a syntax to specify a partition is by using the partition for (DATE) clause in the query: See below example:

partition for (to_date(’15-OCT-2019′,’DD-MON-YYYY’));

SELECT * FROM T_ACCOUNT partition  for (to_date('01-SEP-2019','DD-MON-YYYY'));

The following restrictions apply on Interval Partition:

Only one partitioning key column allowed (NUMBER or DATE)
Minimum one partition must be defined when the table is created.
MAXVALUE partition cannot be defined.
NULL values are not allowed in the partition column.
Can’t be used at the subpartition level.
Not supported for index-organized tables.
You cannot create a domain index on an interval partitioned table.

Daily Partitions Example:

CREATE TABLE T_TRANSACTIONS
  (
    TRANSACTION_KEY NUMBER,
    AMOUNT          NUMBER,
    CUST_ID         NUMBER,
    TRANSACTION_DATE DATE
  )
  PARTITION BY RANGE
  (
    TRANSACTION_DATE
  )
  INTERVAL
  (
    NUMTODSINTERVAL(1, 'day')
  )
  (
    PARTITION P_01012019 VALUES LESS THAN (TO_DATE('01-JAN-2019','DD-MON-YYYY'))
  );

Thanks!

Happy Learning! Your feedback would be appreciated!

Follow @shobhitsinghIN

Oracle Range Partition

Range partitioning is useful when data has some sort of logical range. Example dates or numbers. See below example where monthly range partitions created based on date column in account table.

CREATE TABLE T_ACCOUNT
  (
    V_ACCOUNT_NUM  VARCHAR2(100 CHAR),
    V_ACCOUNT_NAME VARCHAR2(200 CHAR),
    ACC_DATE DATE
  )
  PARTITION BY RANGE
  (
    ACC_DATE
  )
  (
    PARTITION P_DEC2018 VALUES LESS THAN (TO_DATE('01-JAN-2019','DD-MON-YYYY')),
    PARTITION P_JAN2019 VALUES LESS THAN (TO_DATE('01-FEB-2019','DD-MON-YYYY')),
    PARTITION P_FEB2019 VALUES LESS THAN (TO_DATE('01-MAR-2019','DD-MON-YYYY')),
    PARTITION P_MAR2019 VALUES LESS THAN (TO_DATE('01-APR-2019','DD-MON-YYYY'))
  );

Fetching the data from particular partition : In below example you will able to see only match partition data.

SELECT * FROM T_ACCOUNT PARTITION(P_MAR2019);

Check partitions metadata in user_tab_partitions & subpartitions.

SELECT * FROM USER_TAB_PARTITIONS WHERE TABLE_NAME='T_ACCOUNT';

To check the Partition KEY of the table, use below query:

SELECT partition_name,
  column_name,
  high_value,
  partition_position
FROM USER_TAB_PARTITIONS PART ,
  USER_PART_KEY_COLUMNS PART_KEY
WHERE table_name ='T_ACCOUNT'
AND PART.table_name = PART_KEY.name;

In the above example any record having acc_date less than ’01-JAN-2019′ will go to P_DEC2018 partition. See below example:

INSERT
INTO "T_ACCOUNT"
  (
    V_ACCOUNT_NUM,
    V_ACCOUNT_NAME,
    ACC_DATE
  )
  VALUES
  (
    '1234567890',
    'ABC',
    TO_DATE('20-OCT-2010', 'DD-MON-YYYY')
  );
COMMIT;
SELECT * FROM T_ACCOUNT PARTITION (P_DEC2018) ;

In the this example any record having acc_date greater than ’31-MAR-2019′ will error out, as no partition exists for that date.

Example:

INSERT
INTO "T_ACCOUNT"
  (
    V_ACCOUNT_NUM,
    V_ACCOUNT_NAME,
    ACC_DATE
  )
  VALUES
  (
    '1234567890',
    'ABC',
    TO_DATE('20-OCT-2019', 'DD-MON-YYYY')
  );

Error report:
SQL Error: ORA-14400: inserted partition key does not map to any partition
14400. 00000 – “inserted partition key does not map to any partition”
*Cause: An attempt was made to insert a record into, a Range or Composite
Range object, with a concatenated partition key that is beyond
the concatenated partition bound list of the last partition -OR-
An attempt was made to insert a record into a List object with
a partition key that did not match the literal values specified
for any of the partitions.
*Action: Do not insert the key. Or, add a partition capable of accepting
the key, Or add values matching the key to a partition specification

Here in this example we can see the partition key is defined maximum upto 2019-04-01 (Non-inclusive) . But we are trying to insert 20-OCT-2019, which is not getting mapped to any partition, that’s why this error occured as date value which we are trying to insert is T_ACCOUNT not satisfying the partition key range criteria.

Solution: To fix it, you may have to add new partitions or add a Maxvalue partition.

ALTER TABLE T_ACCOUNT ADD partition P_OCT2019 VALUES less than (TO_DATE('01-NOV-2019', 'DD-MON-YYYY'));
ALTER TABLE T_ACCOUNT ADD partition P_MAXVALUE VALUES less than (MAXVALUE);

You can also go with : Oracle Interval Partitioning to resolve this issue, but this not preferable.

Drop Partition:

Once historical data is no longer required for business analysis, after retention period the whole partition can be dropped.

DROP PARTITION P_DEC2018;

Updating Key:

When you are trying to update the partition key column for any record.

UPDATE T_ACCOUNT SET ACC_DATE='20-JAN-2019' WHERE v_account_num='1234567890';

Error report:
SQL Error: ORA-14402: updating partition key column would cause a partition change
14402. 00000 – “updating partition key column would cause a partition change”
*Cause: An UPDATE statement attempted to change the value of a partition
key column causing migration of the row to another partition
*Action: Do not attempt to update a partition key column or make sure that
the new partition key is within the range containing the old
partition key.

You can’t move a row with an update: Row movement disabled, which is the default option.

Solution: We can enable Row movement to partition table T_ACCOUNT. It allows rows to be moved across partitions.

ALTER TABLE T_ACCOUNT ENABLE ROW MOVEMENT;
UPDATE T_ACCOUNT SET ACC_DATE='20-JAN-2019' WHERE v_account_num='1234567890';
COMMIT;

Thanks!

Happy Learning! Your feedback would be appreciated!

Follow @shobhitsinghIN

Python [Post 07] | Tuple

A tuple is a collection which is ordered and unchangeable. It is similar to list, but a tuple is immutable unlike lists which are mutable.

It is created by placing items inside a parentheses (), separated by comma. Parentheses is optional.

# Creating Tuple

empty_tuple = ()
my_tup1 = 'N','S','E','W'
my_tup2 = ('North','South','East','West')

print(type(empty_tuple))
print(my_tup1)
print(my_tup2)

<class ‘tuple’>
(‘N’, ‘S’, ‘E’, ‘W’)
(‘North’, ‘South’, ‘East’, ‘West’)

Other way to create tuple by using the tuple() constructor. See below example.

my_tup3 = tuple((1,'South',5.5,('A','B','C'),[0,1,2]))

print(my_tup3)
print(type(my_tup3))

(1, ‘South’, 5.5, (‘A’, ‘B’, ‘C’), [0, 1, 2])
<class ‘tuple’>

# Creating a tuple having single value

my_tup4 = (4)
print(type(my_tup4)) # But the result is showing int, i.e. this is not a tuple

To create a tuple having single value, Just include comma at last.

my_tup4 = (4,)
print(type(my_tup4)) # Now its a tuple

Some Built-in Python Functions

my_tup5 = 1,2.5,3,4

print(type(my_tup5)) # Type
print(len(my_tup5)) # len Function - Gives the total length of the tuple.
print(max(my_tup5)) # max Function - Returns element from the tuple with maximum value.
print(min(my_tup5)) # min Function - Returns element from the tuple with minimum value.

<class ‘tuple’>
4
4
1

### Accessing Tuple Values – Indexing & Slicing

Use the index operator [] to access an item in a tuple. We can use a : to perform slicing which extracts chunk of more than one element as per input. In slicing last index is not included.

sides = 'East','West','North','South'

# Fetch the first element of list sides

print(sides[0])

East

# Fetch the last element of list sides

print(sides[-1])

South

# fetch elements start from index 1st till last

print(sides[1:])

(‘West’, ‘North’, ‘South’)

# fetch elements upto 2nd index, but 2nd is not included

print(sides [:2])

(‘East’, ‘West’)

# fetch element from 1st index to 3rd index

print(sides [1:3])

(‘West’, ‘North’)

# fetching values in case of nested tuple

my_tup = tuple((1,'South',5.5,('A','B','C'),[0,1,2]))

print(my_tup[3][0])
print(my_tup[3][2])

A
C

### Updating Tuples

Python Tuple is immutable i.e. the elements inside a tuple cannot be changed!

sides = ('East','West','North','South')

# Let's try to change the elements of lists sides

sides[0]= 1

TypeError Traceback (most recent call last)
<ipython-input-14-b1dd30231e62> in <module>()
3 # Let’s try to change the elements of lists sides
4
—-> 5 sides[0]= 1
TypeError: ‘tuple’ object does not support item assignment

### Deleting Tuples

Individual element in tuple cannot be removed. Entire tuple can be removed by using del statement.

# Removing tuple completely

del sides

sides # This will raise exception, as sides tuple removed

NameError Traceback (most recent call last)
<ipython-input-16-7623a01b7793> in <module>()
—-> 1 sides # This will raise exception, as sides tuple removed
NameError: name ‘sides’ is not defined

### Concatenation & Repetition of Tuples

sides = ('East','West','North','South')
modesides = ('North-East','North-East','South-East','South-West')

print(sides)
print(modesides)

(‘East’, ‘West’, ‘North’, ‘South’)
(‘North-East’, ‘North-East’, ‘South-East’, ‘South-West’)

# Concatenation

sides = sides + modesides
print(sides)

(‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’, ‘North-East’, ‘South-East’, ‘South-West’)

# Repetition

sides = sides * 2
print(sides)

(‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’, ‘North-East’, ‘South-East’, ‘South-West’, ‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’, ‘North-East’, ‘South-East’, ‘South-West’)

### Tuple Method

Python includes following tuple methods.

Index() :

Returns the index of the first matched item.

sides = 'EAST','West','North','East','South','East'

print(sides.index('East'))
print(sides.index('West'))

3
1

Count():

Returns the count of number of items passed as an argument.

print(sides.count('East'))

Thanks!

Happy Learning! Your feedback would be appreciated!

shobhitsingh.in
Github Code Link

Follow @shobhitsinghIN

Python [Post 06] | Dictionary

Python dictionary is an un-ordered collection of items. It consists of a key value pairs.

Python dictionary is un-ordered & cannot be sorted.
Value can be almost any Python object.
Value can be accessed by unique key in the dictionary.
Keys must be of an immutable data type. Example – strings, numbers or tuples.

Creating a dictionary is as simple as placing items inside curly braces {} separated by comma. An item has a key and the corresponding value expressed as a pair, key: value.

{‘key1′:’value1′,’key2′:’value2’}

# Examples of Python Dictionary:

# Dictionary having string keys & string value
my_dict0 = { "N": "North", "S": "South", "E": "East", "W": "West" }

# Dictionary having integer keys & string value
my_dict1 = { 1: 'North', 2: 'South', 3: 'East', 4: 'West'}

# Dictionary having string keys & integer value
my_dict2 = { 'North' : 1, 'South': 2, 'East': 3,'West':4}

# Dictionary having mixed keys & values
my_dict3 = { 'N' : 'North', 2: 'South', 'East': 3,'West':'W', "ComplexNumber":5+500j, 5.5:[1,2,3,4]}

# Nested Dictionary
my_dict4 = { 'Aphabets' : { 'A':{'Fruit':'Apple','Object':'Aeroplane'},'B':{'Fruit':'Banana','Object':'Ball'}} ,
'Numbers' : { 0: 'Zero', 1:'One', 2:'Two', 3:'Three', 4:'Four', 5:'Five', 6:'Six', 7:'Seven'}
}
print(my_dict0)
print(my_dict1)
print(my_dict2)
print(my_dict3)
print(my_dict4)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}
{1: ‘North’, 2: ‘South’, 3: ‘East’, 4: ‘West’}
{‘North’: 1, ‘South’: 2, ‘East’: 3, ‘West’: 4}
{‘N’: ‘North’, 2: ‘South’, ‘East’: 3, ‘West’: ‘W’, ‘ComplexNumber’: (5+500j), 5.5: [1, 2, 3, 4]}
{‘Aphabets’: {‘A’: {‘Fruit’: ‘Apple’, ‘Object’: ‘Aeroplane’}, ‘B’: {‘Fruit’: ‘Banana’, ‘Object’: ‘Ball’}}, ‘Numbers’: {0: ‘Zero’, 1: ‘One’, 2: ‘Two’, 3: ‘Three’, 4: ‘Four’, 5: ‘Five’, 6: ‘Six’, 7: ‘Seven’}}

Other way to create dictionary to use the dict() constructor to make a dictionary:

sides = dict(N="North", S="South", E="East",W="West")
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}

Access elements from a dictionary

# Fetching dictionary values by key

print(sides['N'])
print(sides.get('S'))

North
South

# Nested Dictionary
my_dict = { 'Aphabets' : { 'A':{'Fruit':'Apple','Object':'Aeroplane'},'B':{'Fruit':'Banana','Object':'Ball'}} ,
'Numbers' : { 0: 'Zero', 1:'One', 2:'Two', 3:'Three', 4:'Four', 5:'Five', 6:'Six', 7:'Seven'}
}

smy_dict['Aphabets']['A']['Object']

‘Aeroplane’

my_dict4['Numbers'][5]

‘Five’

In Python dictionary no duplicate key is allowed.

# Creating a dictionary having duplicate keys

sides ={ "N": "East", "S": "South", "N": "West", "N": "North" }
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’}

When duplicate keys found during dictionary creation, the last assignment is final one. See above example.

### Updating Dictionary

Adding a new key-value pair, modifying an existing key-value pair, or deleting an existing key-value pair.

# Creating a new dictionary

sides = dict(N="North", S="South", E="East",W="West")
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}

Adding a new key:value pairs to existing dictionary

sides['NE'] = 'North-East'
sides['NW'] = 'North-West'
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’, ‘NE’: ‘North-East’, ‘NW’: ‘North-West’}

Updating key:value pair

sides['NE'] = 'North East'
sides['NW'] = 'North West'
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’, ‘NE’: ‘North East’, ‘NW’: ‘North West’}

sides['NE']

‘North East’

Deleting an existing key:value pair

print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’, ‘NE’: ‘North East’, ‘NW’: ‘North West’}

del sides ['N'] # Remove the provided Key
print(sides)

{‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’, ‘NE’: ‘North East’, ‘NW’: ‘North West’}

del sides ; # delete dictionary completely

### Dictionary Methods

Python Dictionary provides so many methods. See below some examples

sides = dict(N="North", S="South", E="East",W="West")
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}

Get()

It return Value for Key.

print(sides.get('S'))

South

Items(), Keys () & Value()

Item – It return the list with all dictionary keys with values.
Keys – It return list of dictionary dict’s keys.
Value – It returns list of dictionary dict’s values.

print(sides.items())
print(sides.keys())
print(sides.values())

dict_items([(‘N’, ‘North’), (‘S’, ‘South’), (‘E’, ‘East’), (‘W’, ‘West’)])
dict_keys([‘N’, ‘S’, ‘E’, ‘W’])
dict_values([‘North’, ‘South’, ‘East’, ‘West’])

Copy()

It creates the copy of the dictionary into other dictionary.

new_sides = {}
new_sides = sides.copy()

print(new_sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}

Update()

Adds dictionary key-values pairs to another dictionary.

sides = dict(N="North", S="South", E="East",W="West")
some_more_sides = dict(NE="North-East", SE="South-East")

print(sides)
print(some_more_sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’}
{‘NE’: ‘North-East’, ‘SE’: ‘South-East’}

sides.update(some_more_sides)
print(sides)

{‘N’: ‘North’, ‘S’: ‘South’, ‘E’: ‘East’, ‘W’: ‘West’, ‘NE’: ‘North-East’, ‘SE’: ‘South-East’}

Clear()

Removes all elements of dictionary.

some_more_sides.clear(); # Remove all entries in dictionary
print(some_more_sides)

{}

Thanks!

Happy Learning! Your feedback would be appreciated!

shobhitsingh.in

Github Code Link

Follow @shobhitsinghIN

Python [Post 05] | Lists

A list in Python is an ordered group of items. Items in a list need not be of the same type. A list is created by placing all the items inside a square bracket, separated by commas.

# Lists Examples

my_empty_list = []
number_list = [1,2,3,4,5]
str_list = ['A','B','C','D','Hello Python']
mix_list = ['A',1,'B',2,'C',12.5,5+500j]
nested_list = ['A',1,'B',2,['X','Y','Z']]

print(number_list)
print(str_list)
print(mix_list)
print(nested_list)

[1, 2, 3, 4, 5]
[‘A’, ‘B’, ‘C’, ‘D’, ‘Hello Python’]
[‘A’, 1, ‘B’, 2, ‘C’, 12.5, (5+500j)]
[‘A’, 1, ‘B’, 2, [‘X’, ‘Y’, ‘Z’]]

Some Built-in Python Functions

my_list = [1,2.5,3,4.6,5,6]

print(len(my_list)) # len Function - Gives the total length of the list.
print(max(my_list)) # max Function - Returns element from the list with maximum value.
print(min(my_list)) # min Function - Returns element from the list with minimum value.

6
6
1

### List Indexing & Slicing

Use the index operator [] to access an item in a list. We can use a : to perform slicing which extracts chunk of more than one element as per input. In slicing last index is not included.

sides = ['East','West','North','South']

# Fetch the first element of list sides

sides[0]

‘East’

# Fetch the last element of list sides

sides[-1]

‘South’

# fetch elements start from index 1st till last

sides[1:]

[‘West’, ‘North’, ‘South’]

# fetch elements upto 2nd index, but 2nd is not included

sides [:2]

[‘East’, ‘West’]

# fetch element from 1st index to 3rd index

sides [1:3]

[‘West’, ‘North’]

Unlike Python Strings, Python Lists are mutable i.e. the elements inside a list can be changed!

sides = ['East','West','North','South']
print(sides)

# Let's try to change the elements of lists sides

sides[0]= 3.0
sides[1]='Hello World'
sides[2]='Python World'
sides[3]= 12345

print(sides)

[‘East’, ‘West’, ‘North’, ‘South’]
[3.0, ‘Hello World’, ‘Python World’, 12345]

Nested Lists

A list inside a list.

some_sides = ['East','West','North','South']
more_sides1 = ['North-East','North-West']
more_sides2 = ['South-East','Soth-West']

nested_sides = [some_sides,more_sides1,more_sides2]

print(nested_sides)

[[‘East’, ‘West’, ‘North’, ‘South’], [‘North-East’, ‘North-West’], [‘South-East’, ‘Soth-West’]]

# Fetch the first element of first list

print(nested_sides[0])
print(nested_sides[0][0])

[‘East’, ‘West’, ‘North’, ‘South’]
East

# Fetch the second element of third list

print(nested_sides[2])
print(nested_sides[2][1])

[‘South-East’, ‘Soth-West’]
Soth-West

### Lists Methods

Python includes following list methods

# Creating a new list

sides = ['East','West','North','South']

Append():

Adds single element to the end of a list. Length of the list increases by 1 index.

# Example 1: Append Element in list

sides.append('North-East')
print(sides)

[‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’]

# Other way to append

sides = sides + ['South-East','South-West']
print(sides)

[‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’, ‘South-East’, ‘South-West’]

# Example 2: Append List with List (This will create nested list)

sides = ['East','West']
more_sides = ['North','South','North-East']

sides.append(more_sides)
print(sides)

[‘East’, ‘West’, [‘North’, ‘South’, ‘North-East’]]

Extend():

Add all elements of a list to the another list. It will not create nested list.

# Example - Extending Lists

sides = ['East','West']
more_sides = ['North','South','North-East']

sides.extend(more_sides)
print(sides)

[‘East’, ‘West’, ‘North’, ‘South’, ‘North-East’]

Insert():

Inserts an element at the position mentioned in its arguments. It takes two arguments, position and element.

# Example Insert

sides = ['West','North']

print(sides)

sides.insert(2,'South')
sides.insert(3,'South-East')
sides.insert(0,'East')

print(sides)

[‘West’, ‘North’]
[‘East’, ‘West’, ‘North’, ‘South’, ‘South-East’]

Pop():

Removes and returns an element at the given index. By default pop takes off the last index, but you can also specify which index to pop off.

sides.pop() # Pop off the last indexed item, by default

‘South-East’

print(sides)

[‘East’, ‘West’, ‘North’, ‘South’]

sides.pop(3) # Pop off the 3rd indexed item

‘South’

print(sides)

[‘East’, ‘West’, ‘North’]

Remove():

It is used to delete the first occurrence of item mentioned in its arguments.

sides = ['East','West','North','East','South']

sides.remove('East')
print(sides)

[‘West’, ‘North’, ‘East’, ‘South’]

Clear():

Removes all elements from the list.

sides = ['East','West','North','South']
print(sides)

[‘East’, ‘West’, ‘North’, ‘South’]

sides.clear() # Clear the list

print(len(sides))
print(sides)

0
[]

Index() :

Returns the index of the first matched item.

sides = ['EAST','West','North','East','South','East']

sides.index('East')

Count():

Returns the count of number of items passed as an argument.

sides.count('East')

Sort():

Sort items in a list in ascending order.

sides = ['West','North','East','South']

print(sides)
sides.sort()
print(sides)

[‘West’, ‘North’, ‘East’, ‘South’]
[‘East’, ‘North’, ‘South’, ‘West’]

Reverse():

Reverse the order of items in the list.

print(sides)
sides.reverse()
print(sides)

[‘East’, ‘North’, ‘South’, ‘West’]
[‘West’, ‘South’, ‘North’, ‘East’]

Thanks!

Happy Learning! Your feedback would be appreciated!

shobhitsingh.in

Github Code

Follow @shobhitsinghIN

In Data We Trust!

Cloud Concept & Technology

Security & Compliance

Billing & Pricing

Monthly Partition:

Daily Partitions Example:

Drop Partition:

Updating Key:

Some Built-in Python Functions

### Accessing Tuple Values – Indexing & Slicing

### Updating Tuples

### Deleting Tuples

### Concatenation & Repetition of Tuples

### Tuple Method

Index() :

Count():

Access elements from a dictionary

In Python dictionary no duplicate key is allowed.

### Updating Dictionary

Adding a new key:value pairs to existing dictionary

Updating key:value pair

Deleting an existing key:value pair

### Dictionary Methods

Get()

Items(), Keys () & Value()

Copy()

Update()

Clear()

Some Built-in Python Functions

### List Indexing & Slicing

Nested Lists

### Lists Methods

Append():

Extend():

Insert():

Pop():

Remove():

Clear():

Index() :

Count():

Sort():

Reverse():