Big data is a very large amount of structured or unstructured data. This data is so “big”. That it gets problematic to using conventional database techniques and software. Who handles evaluating large amounts of data. The most important task of a Big Data Scientist is bulk data understood.
The term “Big data” used the tools and procedures an organization to process a large volume of data. That around 91% of the data today has created in the last 3 years. The need for data handling has led to a need for developing and using Big Data Technologies.
Clear examples of Big Data could be:
Around 600 million tweets sent in a day. This is more than 6,840 tweets per second.
VISA handles around 172,800,000 card transactions every day.
The 3 Vs in Big Data
Which are Variety, Velocity and Volume.
The varying formats of data. Like databases, excel sheets, documents and several existing formats.
The rate which with the data keeps changing, or in other words the rate at which data create and update.
The size of the available data. Today, data size has become enormous, ranging from gigabytes to even petabytes.
Skills Required Becoming a Big Data Scientist
A set of technical skills, visualization skills and business domain expertise. A data scientist should also have strong analytical and problem-solving skills.
1. Technical Skills:
Knowledge of at least one big data technology such as Hadoop.
Knowledge of programming and scripting languages like Java and Python.
Knowledge of database management and SQL.
Knowledge of data modeling and relational databases.
Knowledge of statistical tools like SAS and Excel.
2. Visualization Skills:
These include presentation skills and knowledge of tools. Like PowerPoint, Google Visualization API, Tableau, MS Paint etc.
3. Business Skills:
These include knowledge of the business domain. Where going to work understanding and knowledge of risk analysis etc.
Big data has many applications for the capital market companies
Exploring data
Finding and managing data for every enterprise data in the big challenge. Big Data technologies can help these enterprises in exploring the “Big Data".
Risk Analytics
Risks, frauds and security controlled by using big data technologies. This could benefit in banking, insurance etc.
Trading Analytics
Companies can analyze their customers. Their needs by using big data technologies for data processing.
Medical Data Management
Big data can help the patient data in the medical sector.
Telecom Data Management
Big data use to decrease the processing time. But managing the call data in the telecom sector. It optimizes the locations based on telecom services.
Financial data management
Financial services companies process several millions of transactions for every day. Big data technologies can help such companies in managing such a big data.
Tax Compliance
Big data could help in detecting tax-related frauds.
Data tagging
Big data can help in organize information of data