Streaming csv parser inspired by binary-csv that aims to be faster than everyone else

A list of practical projects that anyone can solve in any programming language (See solutions). These projects are divided into multiple categories, and each category has its own folder. To get started, simply fork this repo. CONTRIBUTING See ways of contributing to this repo. You can contribute solutions (will be published in this repo) to existing problems, add new projects, or remove existing ones. Make sure you follow all instructions properly. Solutions You can find implementations of these projects in many other languages by other users in this repo. Credits Problems are motivated by the ones shared at: Martyr2’s Mega Project List Rosetta Code Table of Contents Numbers Classic Algorithms Graph Data Structures Text Networking Classes Threading Web Files Databases Graphics and Multimedia Security Numbers Find PI to the Nth Digit - Enter a number and have the program generate PI up to that many decimal places. Keep a limit to how far the program will go. Find e to the Nth Digit - Just like the previous problem, but with e instead of PI. Enter a number and have the program generate e up to that many decimal places. Keep a limit to how far the program will go. Fibonacci Sequence - Enter a number and have the program generate the Fibonacci sequence to that number or to the Nth number. Prime Factorization - Have the user enter a number and find all Prime Factors (if there are any) and display them. Next Prime Number - Have the program find prime numbers until the user chooses to stop asking for the next one. Find Cost of Tile to Cover W x H Floor - Calculate the total cost of the tile it would take to cover a floor plan of width and height, using a cost entered by the user. Mortgage Calculator - Calculate the monthly payments of a fixed-term mortgage over given Nth terms at a given interest rate. Also, figure out how long it will take the user to pay back the loan. For added complexity, add an option for users to select the compounding interval (Monthly, Weekly, Daily, Continually). Change Return Program - The user enters a cost and then the amount of money given. The program will figure out the change and the number of quarters, dimes, nickels, pennies needed for the change. Binary to Decimal and Back Converter - Develop a converter to convert a decimal number to binary or a binary number to its decimal equivalent. Calculator - A simple calculator to do basic operators. Make it a scientific calculator for added complexity. Unit Converter (temp, currency, volume, mass, and more) - Converts various units between one another. The user enters the type of unit being entered, the type of unit they want to convert to, and then the value. The program will then make the conversion. Alarm Clock - A simple clock where it plays a sound after X number of minutes/seconds or at a particular time. Distance Between Two Cities - Calculates the distance between two cities and allows the user to specify a unit of distance. This program may require finding coordinates for the cities like latitude and longitude. Credit Card Validator - Takes in a credit card number from a common credit card vendor (Visa, MasterCard, American Express, Discoverer) and validates it to make sure that it is a valid number (look into how credit cards use a checksum). Tax Calculator - Asks the user to enter a cost and either a country or state tax. It then returns the tax plus the total cost with tax. Factorial Finder - The Factorial of a positive integer, n, is defined as the product of the sequence n, n-1, n-2, ...1, and the factorial of zero, 0, is defined as being 1. Solve this using both loops and recursion. Complex Number Algebra - Show addition, multiplication, negation, and inversion of complex numbers in separate functions. (Subtraction and division operations can be made with pairs of these operations.) Print the results for each operation tested. Happy Numbers - A happy number is defined by the following process. Starting with any positive integer, replace the number by the sum of the squares of its digits, and repeat the process until the number equals 1 (where it will stay), or it loops endlessly in a cycle which does not include 1. Those numbers for which this process ends in 1 are happy numbers, while those that do not end in 1 are unhappy numbers. Display an example of your output here. Find the first 8 happy numbers. Number Names - Show how to spell out a number in English. You can use a preexisting implementation or roll your own, but you should support inputs up to at least one million (or the maximum value of your language's default bounded integer type if that's less). Optional: Support for inputs other than positive integers (like zero, negative integers, and floating-point numbers). Coin Flip Simulation - Write some code that simulates flipping a single coin however many times the user decides. The code should record the outcomes and count the number of tails and heads. Limit Calculator - Ask the user to enter f(x) and the limit value, then return the value of the limit statement Optional: Make the calculator capable of supporting infinite limits. Fast Exponentiation - Ask the user to enter 2 integers a and b and output a^b (i.e. pow(a,b)) in O(LG n) time complexity. Classic Algorithms Collatz Conjecture - Start with a number n > 1. Find the number of steps it takes to reach one using the following process: If n is even, divide it by 2. If n is odd, multiply it by 3 and add 1. Sorting - Implement two types of sorting algorithms: Merge sort and bubble sort. Closest pair problem - The closest pair of points problem or closest pair problem is a problem of computational geometry: given n points in metric space, find a pair of points with the smallest distance between them. Sieve of Eratosthenes - The sieve of Eratosthenes is one of the most efficient ways to find all of the smaller primes (below 10 million or so). Graph Graph from links - Create a program that will create a graph or network from a series of links. Eulerian Path - Create a program that will take as an input a graph and output either an Eulerian path or an Eulerian cycle, or state that it is not possible. An Eulerian path starts at one node and traverses every edge of a graph through every node and finishes at another node. An Eulerian cycle is an eulerian Path that starts and finishes at the same node. Connected Graph - Create a program that takes a graph as an input and outputs whether every node is connected or not. Dijkstra’s Algorithm - Create a program that finds the shortest path through a graph using its edges. Minimum Spanning Tree - Create a program that takes a connected, undirected graph with weights and outputs the minimum spanning tree of the graph i.e., a subgraph that is a tree, contains all the vertices, and the sum of its weights is the least possible. Data Structures Inverted index - An Inverted Index is a data structure used to create full-text search. Given a set of text files, implement a program to create an inverted index. Also, create a user interface to do a search using that inverted index which returns a list of files that contain the query term/terms. The search index can be in memory. Text Fizz Buzz - Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”. Reverse a String - Enter a string and the program will reverse it and print it out. Pig Latin - Pig Latin is a game of alterations played in the English language game. To create the Pig Latin form of an English word the initial consonant sound is transposed to the end of the word and an ay is affixed (Ex.: "banana" would yield anana-bay). Read Wikipedia for more information on rules. Count Vowels - Enter a string and the program counts the number of vowels in the text. For added complexity have it report a sum of each vowel found. Check if Palindrome - Checks if the string entered by the user is a palindrome. That is that it reads the same forwards as backward like “racecar” Count Words in a String - Counts the number of individual words in a string. For added complexity read these strings in from a text file and generate a summary. Text Editor - Notepad-style application that can open, edit, and save text documents. Optional: Add syntax highlighting and other features. RSS Feed Creator - Given a link to RSS/Atom Feed, get all posts and display them. Quote Tracker (market symbols etc) - A program that can go out and check the current value of stocks for a list of symbols entered by the user. The user can set how often the stocks are checked. For CLI, show whether the stock has moved up or down. Optional: If GUI, the program can show green up and red down arrows to show which direction the stock value has moved. Guestbook / Journal - A simple application that allows people to add comments or write journal entries. It can allow comments or not and timestamps for all entries. Could also be made into a shoutbox. Optional: Deploy it on Google App Engine or Heroku or any other PaaS (if possible, of course). Vigenere / Vernam / Ceasar Ciphers - Functions for encrypting and decrypting data messages. Then send them to a friend. Regex Query Tool - A tool that allows the user to enter a text string and then in a separate control enter a regex pattern. It will run the regular expression against the source text and return any matches or flag errors in the regular expression. Networking FTP Program - A file transfer program that can transfer files back and forth from a remote web sever. Bandwidth Monitor - A small utility program that tracks how much data you have uploaded and downloaded from the net during the course of your current online session. See if you can find out what periods of the day you use more and less and generate a report or graph that shows it. Port Scanner - Enter an IP address and a port range where the program will then attempt to find open ports on the given computer by connecting to each of them. On any successful connections mark the port as open. Mail Checker (POP3 / IMAP) - The user enters various account information include web server and IP, protocol type (POP3 or IMAP), and the application will check for email at a given interval. Country from IP Lookup - Enter an IP address and find the country that IP is registered in. Optional: Find the Ip automatically. Whois Search Tool - Enter an IP or host address and have it look it up through whois and return the results to you. Site Checker with Time Scheduling - An application that attempts to connect to a website or server every so many minute or a given time and check if it is up. If it is down, it will notify you by email or by posting a notice on the screen. Classes Product Inventory Project - Create an application that manages an inventory of products. Create a product class that has a price, id, and quantity on hand. Then create an inventory class that keeps track of various products and can sum up the inventory value. Airline / Hotel Reservation System - Create a reservation system that books airline seats or hotel rooms. It charges various rates for particular sections of the plane or hotel. For example, first class is going to cost more than a coach. Hotel rooms have penthouse suites which cost more. Keep track of when rooms will be available and can be scheduled. Company Manager - Create a hierarchy of classes - abstract class Employee and subclasses HourlyEmployee, SalariedEmployee, Manager, and Executive. Everyone's pay is calculated differently, research a bit about it. After you've established an employee hierarchy, create a Company class that allows you to manage the employees. You should be able to hire, fire, and raise employees. Bank Account Manager - Create a class called Account which will be an abstract class for three other classes called CheckingAccount, SavingsAccount, and BusinessAccount. Manage credits and debits from these accounts through an ATM-style program. Patient / Doctor Scheduler - Create a patient class and a doctor class. Have a doctor that can handle multiple patients and set up a scheduling program where a doctor can only handle 16 patients during an 8 hr workday. Recipe Creator and Manager - Create a recipe class with ingredients and put them in a recipe manager program that organizes them into categories like desserts, main courses, or by ingredients like chicken, beef, soups, pies, etc. Image Gallery - Create an image abstract class and then a class that inherits from it for each image type. Put them in a program that displays them in a gallery-style format for viewing. Shape Area and Perimeter Classes - Create an abstract class called Shape and then inherit from it other shapes like diamond, rectangle, circle, triangle, etc. Then have each class override the area and perimeter functionality to handle each shape type. Flower Shop Ordering To Go - Create a flower shop application that deals in flower objects and use those flower objects in a bouquet object which can then be sold. Keep track of the number of objects and when you may need to order more. Family Tree Creator - Create a class called Person which will have a name, when they were born, and when (and if) they died. Allow the user to create these Person classes and put them into a family tree structure. Print out the tree to the screen. Threading Create A Progress Bar for Downloads - Create a progress bar for applications that can keep track of a download in progress. The progress bar will be on a separate thread and will communicate with the main thread using delegates. Bulk Thumbnail Creator - Picture processing can take a bit of time for some transformations. Especially if the image is large. Create an image program that can take hundreds of images and converts them to a specified size in the background thread while you do other things. For added complexity, have one thread handling re-sizing, have another bulk renaming of thumbnails, etc. Web Page Scraper - Create an application that connects to a site and pulls out all links, or images, and saves them to a list. Optional: Organize the indexed content and don’t allow duplicates. Have it put the results into an easily searchable index file. Online White Board - Create an application that allows you to draw pictures, write notes and use various colors to flesh out ideas for projects. Optional: Add a feature to invite friends to collaborate on a whiteboard online. Get Atomic Time from Internet Clock - This program will get the true atomic time from an atomic time clock on the Internet. Use any one of the atomic clocks returned by a simple Google search. Fetch Current Weather - Get the current weather for a given zip/postal code. Optional: Try locating the user automatically. Scheduled Auto Login and Action - Make an application that logs into a given site on a schedule and invokes a certain action and then logs out. This can be useful for checking webmail, posting regular content, or getting info for other applications and saving it to your computer. E-Card Generator - Make a site that allows people to generate their own little e-cards and send them to other people. Do not use Flash. Use a picture library and perhaps insightful mottos or quotes. Content Management System - Create a content management system (CMS) like Joomla, Drupal, PHP Nuke, etc. Start small. Optional: Allow for the addition of modules/addons. Web Board (Forum) - Create a forum for you and your buddies to post, administer and share thoughts and ideas. CAPTCHA Maker - Ever see those images with letters numbers when you signup for a service and then ask you to enter what you see? It keeps web bots from automatically signing up and spamming. Try creating one yourself for online forms. Files Quiz Maker - Make an application that takes various questions from a file, picked randomly, and puts together a quiz for students. Each quiz can be different and then reads a key to grade the quizzes. Sort Excel/CSV File Utility - Reads a file of records, sorts them, and then writes them back to the file. Allow the user to choose various sort style and sorting based on a particular field. Create Zip File Maker - The user enters various files from different directories and the program zips them up into a zip file. Optional: Apply actual compression to the files. Start with Huffman Algorithm. PDF Generator - An application that can read in a text file, HTML file, or some other file and generates a PDF file out of it. Great for a web-based service where the user uploads the file and the program returns a PDF of the file. Optional: Deploy on GAE or Heroku if possible. Mp3 Tagger - Modify and add ID3v1 tags to MP3 files. See if you can also add in the album art into the MP3 file’s header as well as other ID3v2 tags. Code Snippet Manager - Another utility program that allows coders to put in functions, classes, or other tidbits to save for use later. Organized by the type of snippet or language the coder can quickly lookup code. Optional: For extra practice try adding syntax highlighting based on the language. Databases SQL Query Analyzer - A utility application in which a user can enter a query and have it run against a local database and look for ways to make it more efficient. Remote SQL Tool - A utility that can execute queries on remote servers from your local computer across the Internet. It should take in a remote host, user name, and password, run the query and return the results. Report Generator - Create a utility that generates a report based on some tables in a database. Generates sales reports based on the order/order details tables or sums up the day's current database activity. Event Scheduler and Calendar - Make an application that allows the user to enter a date and time of an event, event notes, and then schedule those events on a calendar. The user can then browse the calendar or search the calendar for specific events. Optional: Allow the application to create re-occurrence events that reoccur every day, week, month, year, etc. Budget Tracker - Write an application that keeps track of a household’s budget. The user can add expenses, income, and recurring costs to find out how much they are saving or losing over a period of time. Optional: Allow the user to specify a date range and see the net flow of money in and out of the house budget for that time period. TV Show Tracker - Got a favorite show you don’t want to miss? Don’t have a PVR or want to be able to find the show to then PVR it later? Make an application that can search various online TV Guide sites, locate the shows/times/channels and add them to a database application. The database/website then can send you email reminders that a show is about to start and which channel it will be on. Travel Planner System - Make a system that allows users to put together their own little travel itinerary and keep track of the airline/hotel arrangements, points of interest, budget, and schedule. Graphics and Multimedia Slide Show - Make an application that shows various pictures in a slide show format. Optional: Try adding various effects like fade in/out, star wipe, and window blinds transitions. Stream Video from Online - Try to create your own online streaming video player. Mp3 Player - A simple program for playing your favorite music files. Add features you think are missing from your favorite music player. Watermarking Application - Have some pictures you want copyright protected? Add your own logo or text lightly across the background so that no one can simply steal your graphics off your site. Make a program that will add this watermark to the picture. Optional: Use threading to process multiple images simultaneously. Turtle Graphics - This is a common project where you create a floor of 20 x 20 squares. Using various commands you tell a turtle to draw a line on the floor. You have moved forward, left or right, lift or drop the pen, etc. Do a search online for "Turtle Graphics" for more information. Optional: Allow the program to read in the list of commands from a file. GIF Creator A program that puts together multiple images (PNGs, JPGs, TIFFs) to make a smooth GIF that can be exported. Optional: Make the program convert small video files to GIFs as well. Security Caesar cipher - Implement a Caesar cipher, both encoding, and decoding. The key is an integer from 1 to 25. This cipher rotates the letters of the alphabet (A to Z). The encoding replaces each letter with the 1st to 25th next letter in the alphabet (wrapping Z to A). So key 2 encrypts "HI" to "JK", but key 20 encrypts "HI" to "BC". This simple "monoalphabetic substitution cipher" provides almost no security, because an attacker who has the encoded message can either use frequency analysis to guess the key, or just try all 25 keys.

W3C Strategic Highlights September 2019 This report was prepared for the September 2019 W3C Advisory Committee Meeting (W3C Member link). See the accompanying W3C Fact Sheet — September 2019. For the previous edition, see the April 2019 W3C Strategic Highlights. For future editions of this report, please consult the latest version. A Chinese translation is available. ☰ Contents Introduction Future Web Standards Meeting Industry Needs Web Payments Digital Publishing Media and Entertainment Web & Telecommunications Real-Time Communications (WebRTC) Web & Networks Automotive Web of Things Strengthening the Core of the Web HTML CSS Fonts SVG Audio Performance Web Performance WebAssembly Testing Browser Testing and Tools WebPlatform Tests Web of Data Web for All Security, Privacy, Identity Internationalization (i18n) Web Accessibility Outreach to the world W3C Developer Relations W3C Training Translations W3C Liaisons Introduction This report highlights recent work of enhancement of the existing landscape of the Web platform and innovation for the growth and strength of the Web. 33 working groups and a dozen interest groups enable W3C to pursue its mission through the creation of Web standards, guidelines, and supporting materials. We track the tremendous work done across the Consortium through homogeneous work-spaces in Github which enables better monitoring and management. We are in the middle of a period where we are chartering numerous working groups which demonstrate the rapid degree of change for the Web platform: After 4 years, we are nearly ready to publish a Payment Request API Proposed Recommendation and we need to soon charter follow-on work. In the last year we chartered the Web Payment Security Interest Group. In the last year we chartered the Web Media Working Group with 7 specifications for next generation Media support on the Web. We have Accessibility Guidelines under W3C Member review which includes Silver, a new approach. We have just launched the Decentralized Identifier Working Group which has tremendous potential because Decentralized Identifier (DID) is an identifier that is globally unique, resolveable with high availability, and cryptographically verifiable. We have Privacy IG (PING) under W3C Member review which strengthens our focus on the tradeoff between privacy and function. We have a new CSS charter under W3C Member review which maps the group's work for the next three years. In this period, W3C and the WHATWG have succesfully completed the negotiation of a Memorandum of Understanding rooted in the mutual belief that that having two distinct specifications claiming to be normative is generally harmful for the Web community. The MOU, signed last May, describes how the two organizations are to collaborate on the development of a single authoritative version of the HTML and DOM specifications. W3C subsequently rechartered the HTML Working Group to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and for the production of W3C Recommendations from WHATWG Review Drafts. As the Web evolves continuously, some groups are looking for ways for specifications to do so as well. So-called "evergreen recommendations" or "living standards" aim to track continuous development (and maintenance) of features, on a feature-by-feature basis, while getting review and patent commitments. We see the maturation and further development of an incredible number of new technologies coming to the Web. Continued progress in many areas demonstrates the vitality of the W3C and the Web community, as the rest of the report illustrates. Future Web Standards W3C has a variety of mechanisms for listening to what the community thinks could become good future Web standards. These include discussions with the Membership, discussions with other standards bodies, the activities of thousands of participants in over 300 community groups, and W3C Workshops. There are lots of good ideas. The W3C strategy team has been identifying promising topics and invites public participation. Future, recent and under consideration Workshops include: Inclusive XR (5-6 November 2019, Seattle, WA, USA) to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive, including to people with disabilities; W3C Workshop on Data Models for Transportation (12-13 September 2019, Palo Alto, CA, USA) W3C Workshop on Web Games (27-28 June 2019, Redmond, WA, USA), view report Second W3C Workshop on the Web of Things (3-5 June 2019, Munich, Germany) W3C Workshop on Web Standardization for Graph Data; Creating Bridges: RDF, Property Graph and SQL (4-6 March 2019, Berlin, Germany), view report Web & Machine Learning. The Strategy Funnel documents the staff's exploration of potential new work at various phases: Exploration and Investigation, Incubation and Evaluation, and eventually to the chartering of a new standards group. The Funnel view is a GitHub Project where new area are issues represented by “cards” which move through the columns, usually from left to right. Most cards start in Exploration and move towards Chartering, or move out of the funnel. Public input is welcome at any stage but particularly once Incubation has begun. This helps W3C identify work that is sufficiently incubated to warrant standardization, to review the ecosystem around the work and indicate interest in participating in its standardization, and then to draft a charter that reflects an appropriate scope. Ongoing feedback can speed up the overall standardization process. Since the previous highlights document, W3C has chartered a number of groups, and started discussion on many more: Newly Chartered or Rechartered Web Application Security WG (03-Apr) Web Payment Security IG (17-Apr) Patent and Standards IG (24-Apr) Web Applications WG (14-May) Web & Networks IG (16-May) Media WG (23-May) Media and Entertainment IG (06-Jun) HTML WG (06-Jun) Decentralized Identifier WG (05-Sep) Extended Privacy IG (PING) (30-Sep) Verifiable Claims WG (30-Sep) Service Workers WG (31-Dec) Dataset Exchange WG (31-Dec) Web of Things Working Group (31-Dec) Web Audio Working Group (31-Dec) Proposed charters / Advance Notice Accessibility Guidelines WG Privacy IG (PING) RDF Literal Direction WG Timed Text WG CSS WG Web Authentication WG Closed Internationalization Tag Set IG Meeting Industry Needs Web Payments All Web Payments specifications W3C's payments standards enable a streamlined checkout experience, enabling a consistent user experience across the Web with lower front end development costs for merchants. Users can store and reuse information and more quickly and accurately complete online transactions. The Web Payments Working Group has republished Payment Request API as a Candidate Recommendation, aiming to publish a Proposed Recommendation in the Fall 2019, and is discussing use cases and features for Payment Request after publication of the 1.0 Recommendation. Browser vendors have been finalizing implementation of features added in the past year (view the implementation report). As work continues on the Payment Handler API and its implementation (currently in Chrome and Edge Canary), one focus in 2019 is to increase adoption in other browsers. Recently, Mastercard demonstrated the use of Payment Request API to carry out EMVCo's Secure Remote Commerce (SRC) protocol whose payment method definition is being developed with active participation by Visa, Mastercard, American Express, and Discover. Payment method availability is a key factor in merchant considerations about adopting Payment Request API. The ability to get uniform adoption of a new payment method such as Secure Remote Commerce (SRC) also depends on the availability of the Payment Handler API in browsers, or of proprietary alternatives. Web Monetization, which the Web Payments Working Group will discuss again at its face-to-face meeting in September, can be used to enable micropayments as an alternative revenue stream to advertising. Since the beginning of 2019, Amazon, Brave Software, JCB, Certus Cybersecurity Solutions and Netflix have joined the Web Payments Working Group. In April, W3C launched the Web Payment Security Group to enable W3C, EMVCo, and the FIDO Alliance to collaborate on a vision for Web payment security and interoperability. Participants will define areas of collaboration and identify gaps between existing technical specifications in order to increase compatibility among different technologies, such as: How do SRC, FIDO, and Payment Request relate? The Payment Services Directive 2 (PSD2) regulations in Europe are scheduled to take effect in September 2019. What is the role of EMVCo, W3C, and FIDO technologies, and what is the current state of readiness for the deadline? How can we improve privacy on the Web at the same time as we meet industry requirements regarding user identity? Digital Publishing All Digital Publishing specifications, Publication milestones The Web is the universal publishing platform. Publishing is increasingly impacted by the Web, and the Web increasingly impacts Publishing. Topic of particular interest to Publishing@W3C include typography and layout, accessibility, usability, portability, distribution, archiving, offline access, print on demand, and reliable cross referencing. And the diverse publishing community represented in the groups consist of the traditional "trade" publishers, ebook reading system manufacturers, but also publishers of audio book, scholarly journals or educational materials, library scientists or browser developers. The Publishing Working Group currently concentrates on Audiobooks which lack a comprehensive standard, thus incurring extra costs and time to publish in this booming market. Active development is ongoing on the future standard: Publication Manifest Audiobook profile for Web Publications Lightweight Packaging Format The BD Comics Manga Community Group, the Synchronized Multimedia for Publications Community Group, the Publishing Community Group and a future group on archival, are companions to the working group where specific work is developed and incubated. The Publishing Community Group is a recently launched incubation channel for Publishing@W3C. The goal of the group is to propose, document, and prototype features broadly related to: publications on the Web reading modes and systems and the user experience of publications The EPUB 3 Community Group has successfully completed the revision of EPUB 3.2. The Publishing Business Group fosters ongoing participation by members of the publishing industry and the overall ecosystem in the development of Web infrastructure to better support the needs of the industry. The Business Group serves as an additional conduit to the Publishing Working Group and several Community Groups for feedback between the publishing ecosystem and W3C. The Publishing BG has played a vital role in fostering and advancing the adoption and continued development of EPUB 3. In particular the BG provided critical support to the update of EPUBCheck to validate EPUB content to the new EPUB 3.2 specification. This resulted in the development, in conjunction with the EPUB3 Community Group, of a new generation of EPUBCheck, i.e., EPUBCheck 4.2 production-ready release. Media and Entertainment All Media specifications The Media and Entertainment vertical tracks media-related topics and features that create immersive experiences for end users. HTML5 brought standard audio and video elements to the Web. Standardization activities since then have aimed at turning the Web into a professional platform fully suitable for the delivery of media content and associated materials, enabling missing features to stream video content on the Web such as adaptive streaming and content protection. Together with Microsoft, Comcast, Netflix and Google, W3C received an Technology & Engineering Emmy Award in April 2019 for standardization of a full TV experience on the Web. Current goals are to: Reinforce core media technologies: Creation of the Media Working Group, to develop media-related specifications incubated in the WICG (e.g. Media Capabilities, Picture-in-picture, Media Session) and maintain maintain/evolve Media Source Extensions (MSE) and Encrypted Media Extensions (EME). Improve support for Media Timed Events: data cues incubation. Enhance color support (HDR, wide gamut), in scope of the CSS WG and in the Color on the Web CG. Reduce fragmentation: Continue annual releases of a common and testable baseline media devices, in scope of the Web Media APIs CG and in collaboration with the CTA WAVE Project. Maintain the Road-map of Media Technologies for the Web which highlights Web technologies that can be used to build media applications and services, as well as known gaps to enable additional use cases. Create the future: Discuss perspectives for Media and Entertainment for the Web. Bring the power of GPUs to the Web (graphics, machine learning, heavy processing), under incubation in the GPU for the Web CG. Transition to a Working Group is under discussion. Determine next steps after the successful W3C Workshop on Web Games of June 2019. View the report. Timed Text The Timed Text Working Group develops and maintains formats used for the representation of text synchronized with other timed media, like audio and video, and notably works on TTML, profiles of TTML, and WebVTT. Recent progress includes: A robust WebVTT implementation report poises the specification for publication as a proposed recommendation. Discussions around re-chartering, notably to add a TTML Profile for Audio Description deliverable to the scope of the group, and clarify that rendering of captions within XR content is also in scope. Immersive Web Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The Immersive Web Working Group has been stabilizing the WebXR Device API while the companion Immersive Web Community Group incubates the next series of features identified as key for the future of the Immersive Web. W3C plans a workshop focused on the needs and benefits at the intersection of VR & Accessibility (Inclusive XR), on 5-6 November 2019 in Seattle, WA, USA, to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive. Web & Telecommunications The Web is the Open Platform for Mobile. Telecommunication service providers and network equipment providers have long been critical actors in the deployment of Web technologies. As the Web platform matures, it brings richer and richer capabilities to extend existing services to new users and devices, and propose new and innovative services. Real-Time Communications (WebRTC) All Real-Time Communications specifications WebRTC has reshaped the whole communication landscape by making any connected device a potential communication end-point, bringing audio and video communications anywhere, on any network, vastly expanding the ability of operators to reach their customers. WebRTC serves as the corner-stone of many online communication and collaboration services. The WebRTC Working Group aims to bringing WebRTC 1.0 (and companion specification Media Capture and Streams) to Recommendation by the end of 2019. Intense efforts are focused on testing (supported by a dedicated hackathon at IETF 104) and interoperability. The group is considering pushing features that have not gotten enough traction to separate modules or to a later minor revision of the spec. Beyond WebRTC 1.0, the WebRTC Working Group will focus its efforts on WebRTC NV which the group has started documenting by identifying use cases. Web & Networks Recently launched, in the wake of the May 2018 Web5G workshop, the Web & Networks Interest Group is chaired by representatives from AT&T, China Mobile and Intel, with a goal to explore solutions for web applications to achieve better performance and resource allocation, both on the device and network. The group's first efforts are around use cases, privacy & security requirements and liaisons. Automotive All Automotive specifications To create a rich application ecosystem for vehicles and other devices allowed to connect to the vehicle, the W3C Automotive Working Group is delivering a service specification to expose all common vehicle signals (engine temperature, fuel/charge level, range, tire pressure, speed, etc.) The Vehicle Information Service Specification (VISS), which is a Candidate Recommendation, is seeing more implementations across the industry. It provides the access method to a common data model for all the vehicle signals –presently encapsulating a thousand or so different data elements– and will be growing to accommodate the advances in automotive such as autonomous and driver assist technologies and electrification. The group is already working on a successor to VISS, leveraging the underlying data model and the VIWI submission from Volkswagen, for a more robust means of accessing vehicle signals information and the same paradigm for other automotive needs including location-based services, media, notifications and caching content. The Automotive and Web Platform Business Group acts as an incubator for prospective standards work. One of its task forces is using W3C VISS in performing data sampling and off-boarding the information to the cloud. Access to the wealth of information that W3C's auto signals standard exposes is of interest to regulators, urban planners, insurance companies, auto manufacturers, fleet managers and owners, service providers and others. In addition to components needed for data sampling and edge computing, capturing user and owner consent, information collection methods and handling of data are in scope. The upcoming W3C Workshop on Data Models for Transportation (September 2019) is expected to focus on the need of additional ontologies around transportation space. Web of Things All Web of Things specifications W3C's Web of Things work is designed to bridge disparate technology stacks to allow devices to work together and achieve scale, thus enabling the potential of the Internet of Things by eliminating fragmentation and fostering interoperability. Thing descriptions expressed in JSON-LD cover the behavior, interaction affordances, data schema, security configuration, and protocol bindings. The Web of Things complements existing IoT ecosystems to reduce the cost and risk for suppliers and consumers of applications that create value by combining multiple devices and information services. There are many sectors that will benefit, e.g. smart homes, smart cities, smart industry, smart agriculture, smart healthcare and many more. The Web of Things Working Group is finishing the initial Web of Things standards, with support from the Web of Things Interest Group: Web of Things Architecture Thing Descriptions Strengthening the Core of the Web HTML The HTML Working Group was chartered early June to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and to produce W3C Recommendations from WHATWG Review Drafts. A few days before, W3C and the WHATWG signed a Memorandum of Understanding outlining the agreement to collaborate on the development of a single version of the HTML and DOM specifications. Issues and proposed solutions for HTML and DOM done via the newly rechartered HTML Working Group in the WHATWG repositories The HTML Working Group is targetting November 2019 to bring HTML and DOM to Candidate Recommendations. CSS All CSS specifications CSS is a critical part of the Open Web Platform. The CSS Working Group gathers requirements from two large groups of CSS users: the publishing industry and application developers. Within W3C, those groups are exemplified by the Publishing groups and the Web Platform Working Group. The former requires things like better pagination support and advanced font handling, the latter needs intelligent (and fast!) scrolling and animations. What we know as CSS is actually a collection of almost a hundred specifications, referred to as ‘modules’. The current state of CSS is defined by a snapshot, updated once a year. The group also publishes an index defining every term defined by CSS specifications. Fonts All Fonts specifications The Web Fonts Working Group develops specifications that allow the interoperable deployment of downloadable fonts on the Web, with a focus on Progressive Font Enrichment as well as maintenance of WOFF Recommendations. Recent and ongoing work includes: Early API experiments by Adobe and Monotype have demonstrated the feasibility of a font enrichment API, where a server delivers a font with minimal glyph repertoire and the client can query the full repertoire and request additional subsets on-the-fly. In other experiments, the Brotli compression used in WOFF 2 was extended to support shared dictionaries and patch update. Metrics to quantify improvement are a current hot discussion topic. The group will meet at ATypi 2019 in Japan, to gather requirements from the international typography community. The group will first produce a report summarizing the strengths and weaknesses of each prototype solution by Q2 2020. SVG All SVG specifications SVG is an important and widely-used part of the Open Web Platform. The SVG Working Group focuses on aligning the SVG 2.0 specification with browser implementations, having split the specification into a currently-implemented 2.0 and a forward-looking 2.1. Current activity is on stabilization, increased integration with the Open Web Platform, and test coverage analysis. The Working Group was rechartered in March 2019. A new work item concerns native (non-Web-browser) uses of SVG as a non-interactive, vector graphics format. Audio The Web Audio Working Group was extended to finish its work on the Web Audio API, expecting to publish it as a Recommendation by year end. The specification enables synthesizing audio in the browser. Audio operations are performed with audio nodes, which are linked together to form a modular audio routing graph. Multiple sources — with different types of channel layout — are supported. This modular design provides the flexibility to create complex audio functions with dynamic effects. The first version of Web Audio API is now feature complete and is implemented in all modern browsers. Work has started on the next version, and new features are being incubated in the Audio Community Group. Performance Web Performance All Web Performance specifications There are currently 18 specifications in development in the Web Performance Working Group aiming to provide methods to observe and improve aspects of application performance of user agent features and APIs. The W3C team is looking at related work incubated in the W3C GPU for the Web (WebGPU) Community Group which is poised to transition to a W3C Working Group. A preliminary draft charter is available. WebAssembly All WebAssembly specifications WebAssembly improves Web performance and power by being a virtual machine and execution environment enabling loaded pages to run native (compiled) code. It is deployed in Firefox, Edge, Safari and Chrome. The specification will soon reach Candidate Recommendation. WebAssembly enables near-native performance, optimized load time, and perhaps most importantly, a compilation target for existing code bases. While it has a small number of native types, much of the performance increase relative to Javascript derives from its use of consistent typing. WebAssembly leverages decades of optimization for compiled languages and the byte code is optimized for compactness and streaming (the web page starts executing while the rest of the code downloads). Network and API access all occurs through accompanying Javascript libraries -- the security model is identical to that of Javascript. Requirements gathering and language development occur in the Community Group while the Working Group manages test development, community review and progression of specifications on the Recommendation Track. Testing Browser testing plays a critical role in the growth of the Web by: Improving the reliability of Web technology definitions; Improving the quality of implementations of these technologies by helping vendors to detect bugs in their products; Improving the data available to Web developers on known bugs and deficiencies of Web technologies by publishing results of these tests. Browser Testing and Tools The Browser Testing and Tools Working Group is developing WebDriver version 2, having published last year the W3C Recommendation of WebDriver. WebDriver acts as a remote control interface that enables introspection and control of user agents, provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of Web, and emulates the actions of a real person using the browser. WebPlatform Tests The WebPlatform Tests project now provides a mechanism which allows to fully automate tests that previously needed to be run manually: TestDriver. TestDriver enables sending trusted key and mouse events, sending complex series of trusted pointer and key interactions for things like in-content drag-and-drop or pinch zoom, and even file upload. Since 2014 W3C began work on this coordinated open-source effort to build a cross-browser test suite for the Web Platform, which WHATWG, and all major browsers adopted. Web of Data All Data specifications There have been several great success stories around the standardization of data on the web over the past year. Verifiable Claims seems to have significant uptake. It is also significant that the Distributed Identifier WG charter has received numerous favorable reviews, and was just recently launched. JSON-LD has been a major success with the large deployment on Web sites via schema.org. JSON-LD 1.1 completed technical work, about to transition to CR More than 25% of websites today include schema.org data in JSON-LD The Web of Things description is in CR since May, making use of JSON-LD Verifiable Credentials data model is in CR since July, also making use of JSON-LD Continued strong interest in decentralized identifiers Engagement from the TAG with reframing core documents, such as Ethical Web Principles, to include data on the web within their scope Data is increasingly important for all organizations, especially with the rise of IoT and Big Data. W3C has a mature and extensive suite of standards relating to data that were developed over two decades of experience, with plans for further work on making it easier for developers to work with graph data and knowledge graphs. Linked Data is about the use of URIs as names for things, the ability to dereference these URIs to get further information and to include links to other data. There are ever-increasing sources of open Linked Data on the Web, as well as data services that are restricted to the suppliers and consumers of those services. The digital transformation of industry is seeking to exploit advanced digital technologies. This will facilitate businesses to integrate horizontally along the supply and value chains, and vertically from the factory floor to the office floor. W3C is seeking to make it easier to support enterprise-wide data management and governance, reflecting the strategic importance of data to modern businesses. Traditional approaches to data have focused on tabular databases (SQL/RDBMS), Comma Separated Value (CSV) files, and data embedded in PDF documents and spreadsheets. We're now in midst of a major shift to graph data with nodes and labeled directed links between them. Graph data is: Faster than using SQL and associated JOIN operations More favorable to integrating data from heterogeneous sources Better suited to situations where the data model is evolving In the wake of the recent W3C Workshop on Graph Data we are in the process of launching a Graph Standardization Business Group to provide a business perspective with use cases and requirements, to coordinate technical standards work and liaisons with external organizations. Web for All Security, Privacy, Identity All Security specifications, all Privacy specifications Authentication on the Web As the WebAuthn Level 1 W3C Recommendation published last March is seeing wide implementation and adoption of strong cryptographic authentication, work is proceeding on Level 2. The open standard Web API gives native authentication technology built into native platforms, browsers, operating systems (including mobile) and hardware, offering protection against hacking, credential theft, phishing attacks, thus aiming to end the era of passwords as a security construct. You may read more in our March press release. Privacy An increasing number of W3C specifications are benefitting from Privacy and Security review; there are security and privacy aspects to every specification. Early review is essential. Working with the TAG, the Privacy Interest Group has updated the Self-Review Questionnaire: Security and Privacy. Other recent work of the group includes public blogging further to the exploration of anti-patterns in standards and permission prompts. Security The Web Application Security Working Group adopted Feature Policy, aiming to allow developers to selectively enable, disable, or modify the behavior of some of these browser features and APIs within their application; and Fetch Metadata, aiming to provide servers with enough information to make a priori decisions about whether or not to service a request based on the way it was made, and the context in which it will be used. The Web Payment Security Interest Group, launched last April, convenes members from W3C, EMVCo, and the FIDO Alliance to discuss cooperative work to enhance the security and interoperability of Web payments (read more about payments). Internationalization (i18n) All Internationalization specifications, educational articles related to Internationalization, spec developers checklist Only a quarter or so current Web users use English online and that proportion will continue to decrease as the Web reaches more and more communities of limited English proficiency. If the Web is to live up to the "World Wide" portion of its name, and for the Web to truly work for stakeholders all around the world engaging with content in various languages, it must support the needs of worldwide users as they engage with content in the various languages. The growth of epublishing also brings requirements for new features and improved typography on the Web. It is important to ensure the needs of local communities are captured. The W3C Internationalization Initiative was set up to increase in-house resources dedicated to accelerating progress in making the World Wide Web "worldwide" by gathering user requirements, supporting developers, and education & outreach. For an overview of current projects see the i18n radar. W3C's Internationalization efforts progressed on a number of fronts recently: Requirements: New African and European language groups will work on the gap analysis, errata and layout requirements. Gap analysis: Japanese, Devanagari, Bengali, Tamil, Lao, Khmer, Javanese, and Ethiopic updated in the gap-analysis documents. Layout requirements document: notable progress tracked in the Southeast Asian Task Force while work continues on Chinese layout requirements. Developer support: Spec reviews: the i18n WG continues active review of specifications of the WHATWG and other W3C Working Groups. Short review checklist: easy way to begin a self-review to help spec developers understand what aspects of their spec are likely to need attention for internationalization, and points them to more detailed checklists for the relevant topics. It also helps those reviewing specs for i18n issues. Strings on the Web: Language and Direction Metadata lays out issues and discusses potential solutions for passing information about language and direction with strings in JSON or other data formats. The document was rewritten for clarity, and expanded. The group is collaborating with the JSON-LD and Web Publishing groups to develop a plan for updating RDF, JSON-LD and related specifications to handle metadata for base direction of text (bidi). User-friendly test format: a new format was developed for Internationalization Test Suite tests, which displays helpful information about how the test works. This particularly useful because those tests are pointed to by educational materials and gap-analysis documents. Web Platform Tests: a large number of tests in the i18n test suite have been ported to the WPT repository, including: css-counter-styles, css-ruby, css-syntax, css-test, css-text-decor, css-writing-modes, and css-pseudo. Education & outreach: (for all educational materials, see the HTML & CSS Authoring Techniques) Web Accessibility All Accessibility specifications, WAI resources The Web Accessibility Initiative supports W3C's Web for All mission. Recent achievements include: Education and training: Inaccessibility of CAPTCHA updated to bring our analysis and recommendations up to date with CAPTCHA practice today, concluding two years of extensive work and invaluable input from the public (read more on the W3C Blog Learn why your web content and applications should be accessible. The Education and Outreach Working Group has completed revision and updating of the Business Case for Digital Accessibility. Accessibility guidelines: The Accessibility Guidelines Working Group has continued to update WCAG Techniques and Understanding WCAG 2.1; and published a Candidate Recommendation of Accessibility Conformance Testing Rules Format 1.0 to improve inter-rater reliability when evaluating conformance of web content to WCAG An updated charter is being developed to host work on "Silver", the next generation accessibility guidelines (WCAG 2.2) There are accessibility aspects to most specifications. Check your work with the FAST checklist. Outreach to the world W3C Developer Relations To foster the excellent feedback loop between Web Standards development and Web developers, and to grow participation from that diverse community, recent W3C Developer Relations activities include: @w3cdevs tracks the enormous amount of work happening across W3C W3C Track during the Web Conference 2019 in San Francisco Tech videos: W3C published the 2019 Web Games Workshop videos The 16 September 2019 Developer Meetup in Fukuoka, Japan, is open to all and will combine a set of technical demos prepared by W3C groups, and a series of talks on a selected set of W3C technologies and projects W3C is involved with Mozilla, Google, Samsung, Microsoft and Bocoup in the organization of ViewSource 2019 in Amsterdam (read more on the W3C Blog) W3C Training In partnership with EdX, W3C's MOOC training program, W3Cx offers a complete "Front-End Web Developer" (FEWD) professional certificate program that consists of a suite of five courses on the foundational languages that power the Web: HTML5, CSS and JavaScript. We count nearly 900K students from all over the world. Translations Many Web users rely on translations of documents developed at W3C whose official language is English. W3C is extremely grateful to the continuous efforts of its community in ensuring our various deliverables in general, and in our specifications in particular, are made available in other languages, for free, ensuring their exposure to a much more diverse set of readers. Last Spring we developed a more robust system, a new listing of translations of W3C specifications and updated the instructions on how to contribute to our translation efforts. W3C Liaisons Liaisons and coordination with numerous organizations and Standards Development Organizations (SDOs) is crucial for W3C to: make sure standards are interoperable coordinate our respective agenda in Internet governance: W3C participates in ICANN, GIPO, IGF, the I* organizations (ICANN, IETF, ISOC, IAB). ensure at the government liaison level that our standards work is officially recognized when important to our membership so that products based on them (often done by our members) are part of procurement orders. W3C has ARO/PAS status with ISO. W3C participates in the EU MSP and Rolling Plan on Standardization ensure the global set of Web and Internet standards form a compatible stack of technologies, at the technical and policy level (patent regime, fragmentation, use in policy making) promote Standards adoption equally by the industry, the public sector, and the public at large Coralie Mercier, Editor, W3C Marketing & Communications $Id: Overview.html,v 1.60 2019/10/15 12:05:52 coralie Exp $ Copyright © 2019 W3C ® (MIT, ERCIM, Keio, Beihang) Usage policies apply.

Faster CSV for Python

⚡ Production-grade RAG chunking engine powered by Rust. Process GBs of CSV, PDF, JSON, JSONL, DOCX, XLSX, URLs, ETC., in seconds with O(1) memory. 40x faster than LangChain.

Fastest CSV class for MRI Ruby and JRuby. Faster than faster_csv and fasterer-csv.

json to csv conversion. about 10x faster than jq.

SIMD-accelerated CSV parser for Ruby - 5-6x faster than CSV stdlib

A faster implementation fo a CSV parser than FasterCSV

############################################################ # +------------------------------------------------------+ # # | Notes | # # +------------------------------------------------------+ # ############################################################ # If you want to use special characters in this document, such as accented letters, you MUST save the file as UTF-8, not ANSI. # If you receive an error when Essentials loads, ensure that: # - No tabs are present: YAML only allows spaces # - Indents are correct: YAML hierarchy is based entirely on indentation # - You have "escaped" all apostrophes in your text: If you want to write "don't", for example, write "don''t" instead (note the doubled apostrophe) # - Text with symbols is enclosed in single or double quotation marks # If you have problems join the Essentials help support channel: http://tiny.cc/EssentialsChat ############################################################ # +------------------------------------------------------+ # # | Essentials (Global) | # # +------------------------------------------------------+ # ############################################################ # A color code between 0-9 or a-f. Set to 'none' to disable. ops-name-color: 'none' # The character(s) to prefix all nicknames, so that you know they are not true usernames. nickname-prefix: '~' # Disable this if you have any other plugin, that modifies the displayname of a user. change-displayname: true # When this option is enabled, the (tab) player list will be updated with the displayname. # The value of change-displayname (above) has to be true. #change-playerlist: true # When essentialschat.jar isnt used, force essentials to add the prefix and suffix from permission plugins to displayname # This setting is ignored if essentialschat.jar is used, and defaults to 'true' # The value of change-displayname (above) has to be true. # Do not edit this setting unless you know what you are doing! #add-prefix-suffix: false # The delay, in seconds, required between /home, /tp, etc. teleport-cooldown: 5 # The delay, in seconds, before a user actually teleports. If the user moves or gets attacked in this timeframe, the teleport never occurs. teleport-delay: 5 # The delay, in seconds, a player can't be attacked by other players after they have been teleported by a command # This will also prevent the player attacking other players teleport-invulnerability: 4 # The delay, in seconds, required between /heal attempts heal-cooldown: 60 # What to prevent from /i /give # e.g item-spawn-blacklist: 46,11,10 item-spawn-blacklist: # Set this to true if you want permission based item spawn rules # Note: The blacklist above will be ignored then. # Permissions: # - essentials.itemspawn.item-all # - essentials.itemspawn.item-[itemname] # - essentials.itemspawn.item-[itemid] # - essentials.give.item-all # - essentials.give.item-[itemname] # - essentials.give.item-[itemid] # For more information, visit http://wiki.ess3.net/wiki/Command_Reference/ICheat#Item.2FGive permission-based-item-spawn: false # Mob limit on the /spawnmob command per execution spawnmob-limit: 10 # Shall we notify users when using /lightning warn-on-smite: true # motd and rules are now configured in the files motd.txt and rules.txt # When a command conflicts with another plugin, by default, Essentials will try to force the OTHER plugin to take priority. # Commands in this list, will tell Essentials to 'not give up' the command to other plugins. # In this state, which plugin 'wins' appears to be almost random. # # If you have two plugin with the same command and you wish to force Essentials to take over, you need an alias. # To force essentials to take 'god' alias 'god' to 'egod'. # See http://wiki.bukkit.org/Bukkit.yml#aliases for more information overridden-commands: # - god # Disabled commands will be completely unavailable on the server. # Disabling commands here will have no effect on command conflicts. disabled-commands: # - nick # These commands will be shown to players with socialSpy enabled # You can add commands from other plugins you may want to track or # remove commands that are used for something you dont want to spy on socialspy-commands: - msg - w - r - mail - m - t - whisper - emsg - tell - er - reply - ereply - email - action - describe - eme - eaction - edescribe - etell - ewhisper - pm # If you do not wish to use a permission system, you can define a list of 'player perms' below. # This list has no effect if you are using a supported permissions system. # If you are using an unsupported permissions system simply delete this section. # Whitelist the commands and permissions you wish to give players by default (everything else is op only). # These are the permissions without the "essentials." part. player-commands: - afk - afk.auto - back - back.ondeath - balance - balance.others - balancetop - build - chat.color - chat.format - chat.shout - chat.question - clearinventory - compass - depth - delhome - getpos - geoip.show - help - helpop - home - home.others - ignore - info - itemdb - kit - kits.tools - list - mail - mail.send - me - motd - msg - msg.color - nick - near - pay - ping - protect - r - rules - realname - seen - sell - sethome - setxmpp - signs.create.protection - signs.create.trade - signs.break.protection - signs.break.trade - signs.use.balance - signs.use.buy - signs.use.disposal - signs.use.enchant - signs.use.free - signs.use.gamemode - signs.use.heal - signs.use.info - signs.use.kit - signs.use.mail - signs.use.protection - signs.use.repair - signs.use.sell - signs.use.time - signs.use.trade - signs.use.warp - signs.use.weather - spawn - suicide - time - tpa - tpaccept - tpahere - tpdeny - warp - warp.list - world - worth - xmpp # Note: All items MUST be followed by a quantity! # All kit names should be lower case, and will be treated as lower in permissions/costs. # Syntax: - itemID[:DataValue/Durability] Amount [Enchantment:Level].. [itemmeta:value]... # For Item meta information visit http://wiki.ess3.net/wiki/Item_Meta # 'delay' refers to the cooldown between how often you can use each kit, measured in seconds. # For more information, visit http://wiki.ess3.net/wiki/Kits kits: tools: delay: 10 items: - 272 1 - 273 1 - 274 1 - 275 1 dtools: delay: 600 items: - 278 1 efficiency:1 durability:1 fortune:1 name:&4Gigadrill lore:The_drill_that_&npierces|the_heavens - 277 1 digspeed:3 name:Dwarf lore:Diggy|Diggy|Hole - 298 1 color:255,255,255 name:Top_Hat lore:Good_day,_Good_day - 279:780 1 notch: delay: 6000 items: - 397:3 1 player:Notch color: delay: 6000 items: - 387 1 title:&4Book_&9o_&6Colors author:KHobbits lore:Ingame_color_codes book:Colors firework: delay: 6000 items: - 401 1 name:Angry_Creeper color:red fade:green type:creeper power:1 - 401 1 name:StarryNight color:yellow,orange fade:blue type:star effect:trail,twinkle power:1 - 401 2 name:SolarWind color:yellow,orange fade:red shape:large effect:twinkle color:yellow,orange fade:red shape:ball effect:trail color:red,purple fade:pink shape:star effect:trail power:1 # Essentials Sign Control # See http://wiki.ess3.net/wiki/Sign_Tutorial for instructions on how to use these. # To enable signs, remove # symbol. To disable all signs, comment/remove each sign. # Essentials Colored sign support will be enabled when any sign types are enabled. # Color is not an actual sign, it's for enabling using color codes on signs, when the correct permissions are given. enabledSigns: #- color #- balance #- buy #- sell #- trade #- free #- disposal #- warp #- kit #- mail #- enchant #- gamemode #- heal #- info #- spawnmob #- repair #- time #- weather # How many times per second can Essentials signs be interacted with per player. # Values should be between 1-20, 20 being virtually no lag protection. # Lower numbers will reduce the possibility of lag, but may annoy players. sign-use-per-second: 4 # Backup runs a batch/bash command while saving is disabled backup: # Interval in minutes interval: 30 # Unless you add a valid backup command or script here, this feature will be useless. # Use 'save-all' to simply force regular world saving without backup. #command: 'rdiff-backup World1 backups/World1' # Set this true to enable permission per warp. per-warp-permission: false # Sort output of /list command by groups sort-list-by-groups: false # More output to the console debug: false # Set the locale for all messages # If you don't set this, the default locale of the server will be used. # For example, to set language to English, set locale to en, to use the file "messages_en.properties" # Don't forget to remove the # in front of the line # For more information, visit http://wiki.ess3.net/wiki/Locale #locale: en # Turn off god mode when people exit remove-god-on-disconnect: false # Auto-AFK # After this timeout in seconds, the user will be set as afk. # Set to -1 for no timeout. auto-afk: 300 # Auto-AFK Kick # After this timeout in seconds, the user will be kicked from the server. # Set to -1 for no timeout. auto-afk-kick: -1 # Set this to true, if you want to freeze the player, if he is afk. # Other players or monsters can't push him out of afk mode then. # This will also enable temporary god mode for the afk player. # The player has to use the command /afk to leave the afk mode. freeze-afk-players: false # When the player is afk, should he be able to pickup items? # Enable this, when you don't want people idling in mob traps. disable-item-pickup-while-afk: false # This setting controls if a player is marked as active on interaction. # When this setting is false, you will need to manually un-AFK using the /afk command. cancel-afk-on-interact: true # Should we automatically remove afk status when the player moves? # Player will be removed from AFK on chat/command regardless of this setting. # Disable this to reduce server lag. cancel-afk-on-move: true # You can disable the death messages of Minecraft here death-messages: true # Add worlds to this list, if you want to automatically disable god mode there no-god-in-worlds: # - world_nether # Set to true to enable per-world permissions for teleporting between worlds with essentials commands # This applies to /world, /back, /tp[a|o][here|all], but not warps. # Give someone permission to teleport to a world with essentials.worlds.<worldname> # This does not affect the /home command, there is a separate toggle below for this. world-teleport-permissions: false # The number of items given if the quantity parameter is left out in /item or /give. # If this number is below 1, the maximum stack size size is given. If over-sized stacks # are not enabled, any number higher than the maximum stack size results in more than one stack. default-stack-size: -1 # Over-sized stacks are stacks that ignore the normal max stack size. # They can be obtained using /give and /item, if the player has essentials.oversizedstacks permission. # How many items should be in an over-sized stack? oversized-stacksize: 64 # Allow repair of enchanted weapons and armor. # If you set this to false, you can still allow it for certain players using the permission # essentials.repair.enchanted repair-enchanted: true # Allow 'unsafe' enchantments in kits and item spawning. # Warning: Mixing and overleveling some enchantments can cause issues with clients, servers and plugins. unsafe-enchantments: false #Do you want essentials to keep track of previous location for /back in the teleport listener? #If you set this to true any plugin that uses teleport will have the previous location registered. register-back-in-listener: false #Delay to wait before people can cause attack damage after logging in login-attack-delay: 5 #Set the max fly speed, values range from 0.1 to 1.0 max-fly-speed: 0.8 #Set the maximum amount of mail that can be sent within a minute. mails-per-minute: 1000 # Set the maximum time /tempban can be used for in seconds. # Set to -1 to disable, and essentials.tempban.unlimited can be used to override. max-tempban-time: -1 ############################################################ # +------------------------------------------------------+ # # | EssentialsHome | # # +------------------------------------------------------+ # ############################################################ # Allows people to set their bed at daytime update-bed-at-daytime: true # Set to true to enable per-world permissions for using homes to teleport between worlds # This applies to the /home only. # Give someone permission to teleport to a world with essentials.worlds.<worldname> world-home-permissions: false # Allow players to have multiple homes. # Players need essentials.sethome.multiple before they can have more than 1 home, defaults to 'default' below. # Define different amounts of multiple homes for different permissions, e.g. essentials.sethome.multiple.vip # People with essentials.sethome.multiple.unlimited are not limited by these numbers. # For more information, visit http://wiki.ess3.net/wiki/Multihome sethome-multiple: default: 3 # essentials.sethome.multiple.vip vip: 5 # essentials.sethome.multiple.staff staff: 10 # Set timeout in seconds for players to accept tpa before request is cancelled. # Set to 0 for no timeout tpa-accept-cancellation: 120 ############################################################ # +------------------------------------------------------+ # # | EssentialsEco | # # +------------------------------------------------------+ # ############################################################ # For more information, visit http://wiki.ess3.net/wiki/Essentials_Economy # Defines the balance with which new players begin. Defaults to 0. starting-balance: 0 # worth-# defines the value of an item when it is sold to the server via /sell. # These are now defined in worth.yml # Defines the cost to use the given commands PER USE # Some commands like /repair have sub-costs, check the wiki for more information. command-costs: # /example costs $1000 PER USE #example: 1000 # /kit tools costs $1500 PER USE #kit-tools: 1500 # Set this to a currency symbol you want to use. currency-symbol: '$' # Set the maximum amount of money a player can have # The amount is always limited to 10 trillion because of the limitations of a java double max-money: 10000000000000 # Set the minimum amount of money a player can have (must be above the negative of max-money). # Setting this to 0, will disable overdrafts/loans completely. Users need 'essentials.eco.loan' perm to go below 0. min-money: -10000 # Enable this to log all interactions with trade/buy/sell signs and sell command economy-log-enabled: false ############################################################ # +------------------------------------------------------+ # # | EssentialsHelp | # # +------------------------------------------------------+ # ############################################################ # Show other plugins commands in help non-ess-in-help: true # Hide plugins which do not give a permission # You can override a true value here for a single plugin by adding a permission to a user/group. # The individual permission is: essentials.help.<plugin>, anyone with essentials.* or '*' will see all help regardless. # You can use negative permissions to remove access to just a single plugins help if the following is enabled. hide-permissionless-help: true ############################################################ # +------------------------------------------------------+ # # | EssentialsChat | # # +------------------------------------------------------+ # ############################################################ chat: # If EssentialsChat is installed, this will define how far a player's voice travels, in blocks. Set to 0 to make all chat global. # Note that users with the "essentials.chat.spy" permission will hear everything, regardless of this setting. # Users with essentials.chat.shout can override this by prefixing text with an exclamation mark (!) # Users with essentials.chat.question can override this by prefixing text with a question mark (?) # You can add command costs for shout/question by adding chat-shout and chat-question to the command costs section." radius: 0 # Chat formatting can be done in two ways, you can either define a standard format for all chat # Or you can give a group specific chat format, to give some extra variation. # If set to the default chat format which "should" be compatible with ichat. # For more information of chat formatting, check out the wiki: http://wiki.ess3.net/wiki/Chat_Formatting format: '&l{DISPLAYNAME} &3➽ &f&l{MESSAGE}' Dziewczyna: '{DISPLAYNAME} &3➽ &5 {MESSAGE}' #format: '&7[{GROUP}]&r {DISPLAYNAME}&7:&r {MESSAGE}' group-formats: # Default: '{WORLDNAME} {DISPLAYNAME}&7:&r {MESSAGE}' # Admins: '{WORLDNAME} &c[{GROUP}]&r {DISPLAYNAME}&7:&c {MESSAGE}' # If you are using group formats make sure to remove the '#' to allow the setting to be read. ############################################################ # +------------------------------------------------------+ # # | EssentialsProtect | # # +------------------------------------------------------+ # ############################################################ protect: # Database settings for sign/rail protection # mysql or sqlite # We strongly recommend against using mysql here, unless you have a good reason. # Sqlite seems to be faster in almost all cases, and in some cases mysql can be much slower. datatype: 'sqlite' # If you specified MySQL above, you MUST enter the appropriate details here. # If you specified SQLite above, these will be IGNORED. username: 'root' password: 'root' mysqlDb: 'jdbc:mysql://localhost:3306/minecraft' # General physics/behavior modifications prevent: lava-flow: false water-flow: false water-bucket-flow: false fire-spread: true lava-fire-spread: true flint-fire: false lightning-fire-spread: true portal-creation: false tnt-explosion: false tnt-playerdamage: false fireball-explosion: false fireball-fire: false fireball-playerdamage: false witherskull-explosion: false witherskull-playerdamage: false wither-spawnexplosion: false wither-blockreplace: false creeper-explosion: false creeper-playerdamage: false creeper-blockdamage: false enderdragon-blockdamage: true enderman-pickup: false villager-death: false # Monsters won't follow players # permission essentials.protect.entitytarget.bypass disables this entitytarget: false # Prevent the spawning of creatures spawn: creeper: false skeleton: false spider: false giant: false zombie: false slime: false ghast: false pig_zombie: false enderman: false cave_spider: false silverfish: false blaze: false magma_cube: false ender_dragon: false pig: false sheep: false cow: false chicken: false squid: false wolf: false mushroom_cow: false snowman: false ocelot: false iron_golem: false villager: false wither: false bat: false witch: false # Maximum height the creeper should explode. -1 allows them to explode everywhere. # Set prevent.creeper-explosion to true, if you want to disable creeper explosions. creeper: max-height: -1 # Protect various blocks. protect: # Protect all signs signs: false # Prevent users from destroying rails rails: false # Blocks below rails/signs are also protected if the respective rail/sign is protected. # This makes it more difficult to circumvent protection, and should be enabled. # This only has an effect if "rails" or "signs" is also enabled. block-below: true # Prevent placing blocks above protected rails, this is to stop a potential griefing prevent-block-on-rails: false # Store blocks / signs in memory before writing memstore: false # Disable various default physics and behaviors disable: # Should fall damage be disabled? fall: false # Users with the essentials.protect.pvp permission will still be able to attack each other if this is set to true. # They will be unable to attack users without that same permission node. pvp: false # Should drowning damage be disabled? # (Split into two behaviors; generally, you want both set to the same value) drown: false suffocate: false # Should damage via lava be disabled? Items that fall into lava will still burn to a crisp. ;) lavadmg: false # Should arrow damage be disabled projectiles: false # This will disable damage from touching cacti. contactdmg: false # Burn, baby, burn! Should fire damage be disabled? firedmg: false # Should the damage after hit by a lightning be disabled? lightning: false # Should Wither damage be disabled? wither: false # Disable weather options weather: storm: false thunder: false lightning: false ############################################################ # +------------------------------------------------------+ # # | EssentialsAntiBuild | # # +------------------------------------------------------+ # ############################################################ # Disable various default physics and behaviors # For more information, visit http://wiki.ess3.net/wiki/AntiBuild # Should people with build: false in permissions be allowed to build # Set true to disable building for those people # Setting to false means EssentialsAntiBuild will never prevent you from building build: true # Should people with build: false in permissions be allowed to use items # Set true to disable using for those people # Setting to false means EssentialsAntiBuild will never prevent you from using use: true # Should we tell people they are not allowed to build warn-on-build-disallow: true # For which block types would you like to be alerted? # You can find a list of IDs in plugins/Essentials/items.csv after loading Essentials for the first time. # 10 = lava :: 11 = still lava :: 46 = TNT :: 327 = lava bucket alert: on-placement: 10,11,46,327 on-use: 327 on-break: blacklist: # Which blocks should people be prevented from placing placement: 10,11,46,327 # Which items should people be prevented from using usage: 327 # Which blocks should people be prevented from breaking break: # Which blocks should not be pushed by pistons piston: ############################################################ # +------------------------------------------------------+ # # | Essentials Spawn / New Players | # # +------------------------------------------------------+ # ############################################################ newbies: # Should we announce to the server when someone logs in for the first time? # If so, use this format, replacing {DISPLAYNAME} with the player name. # If not, set to '' #announce-format: '' announce-format: '&dWelcome {DISPLAYNAME}&d to the server!' # When we spawn for the first time, which spawnpoint do we use? # Set to "none" if you want to use the spawn point of the world. spawnpoint: newbies # Do we want to give users anything on first join? Set to '' to disable # This kit will be given regardless of cost, and permissions. #kit: '' kit: tools # Set this to lowest, if you want Multiverse to handle the respawning # Set this to high, if you want EssentialsSpawn to handle the respawning # Set this to highest, if you want to force EssentialsSpawn to handle the respawning respawn-listener-priority: high # When users die, should they respawn at their first home or bed, instead of the spawnpoint? respawn-at-home: false # End of File <-- No seriously, you're done with configuration.

[2025] Fast automated Google Maps scraper with stealth mode and interactive CLI. Extracts all reviews, ratings, profile pictures, reviews images, and owner responses using Puppeteer. Features smart target detection, live progress tracking, timestamped runs, and CSV export. 3-5x faster than traditional scrapers.

Google's Brotli is better at compression than the standard gZIP format. This script allows you to pre-compress css, js, svg, csv, json etc. files to facilitate super-fast static content delivery.

Write CSV files 10x faster than clojure/data.csv and work easily with structured data. Built on top of the excellent osiegmar/FastCSV Java library.

A script to search exploitdb via the files.csv from their SVN. Tends to be faster and more accurate than using the web search.

NSE Stock Data Scraper - Full-stack web app for downloading Indian stock market data. Features Node.js/Express backend with direct NSE API integration (10x faster than web scraping) and React frontend. Provides real-time progress tracking, configurable stock lists, and automated CSV export. Perfect for financial analysis and algorithmic trading.

How to share data with a statistician This is a guide for anyone who needs to share data with a statistician or data scientist. The target audiences I have in mind are: Collaborators who need statisticians or data scientists to analyze data for them Students or postdocs in various disciplines looking for consulting advice Junior statistics students whose job it is to collate/clean/wrangle data sets The goals of this guide are to provide some instruction on the best way to share data to avoid the most common pitfalls and sources of delay in the transition from data collection to data analysis. The Leek group works with a large number of collaborators and the number one source of variation in the speed to results is the status of the data when they arrive at the Leek group. Based on my conversations with other statisticians this is true nearly universally. My strong feeling is that statisticians should be able to handle the data in whatever state they arrive. It is important to see the raw data, understand the steps in the processing pipeline, and be able to incorporate hidden sources of variability in one's data analysis. On the other hand, for many data types, the processing steps are well documented and standardized. So the work of converting the data from raw form to directly analyzable form can be performed before calling on a statistician. This can dramatically speed the turnaround time, since the statistician doesn't have to work through all the pre-processing steps first. What you should deliver to the statistician To facilitate the most efficient and timely analysis this is the information you should pass to a statistician: The raw data. A tidy data set A code book describing each variable and its values in the tidy data set. An explicit and exact recipe you used to go from 1 -> 2,3 Let's look at each part of the data package you will transfer. The raw data It is critical that you include the rawest form of the data that you have access to. This ensures that data provenance can be maintained throughout the workflow. Here are some examples of the raw form of data: The strange binary file your measurement machine spits out The unformatted Excel file with 10 worksheets the company you contracted with sent you The complicated JSON data you got from scraping the Twitter API The hand-entered numbers you collected looking through a microscope You know the raw data are in the right format if you: Ran no software on the data Did not modify any of the data values You did not remove any data from the data set You did not summarize the data in any way If you made any modifications of the raw data it is not the raw form of the data. Reporting modified data as raw data is a very common way to slow down the analysis process, since the analyst will often have to do a forensic study of your data to figure out why the raw data looks weird. (Also imagine what would happen if new data arrived?) The tidy data set The general principles of tidy data are laid out by Hadley Wickham in this paper and this video. While both the paper and the video describe tidy data using R, the principles are more generally applicable: Each variable you measure should be in one column Each different observation of that variable should be in a different row There should be one table for each "kind" of variable If you have multiple tables, they should include a column in the table that allows them to be joined or merged While these are the hard and fast rules, there are a number of other things that will make your data set much easier to handle. First is to include a row at the top of each data table/spreadsheet that contains full row names. So if you measured age at diagnosis for patients, you would head that column with the name AgeAtDiagnosis instead of something like ADx or another abbreviation that may be hard for another person to understand. Here is an example of how this would work from genomics. Suppose that for 20 people you have collected gene expression measurements with RNA-sequencing. You have also collected demographic and clinical information about the patients including their age, treatment, and diagnosis. You would have one table/spreadsheet that contains the clinical/demographic information. It would have four columns (patient id, age, treatment, diagnosis) and 21 rows (a row with variable names, then one row for every patient). You would also have one spreadsheet for the summarized genomic data. Usually this type of data is summarized at the level of the number of counts per exon. Suppose you have 100,000 exons, then you would have a table/spreadsheet that had 21 rows (a row for gene names, and one row for each patient) and 100,001 columns (one row for patient ids and one row for each data type). If you are sharing your data with the collaborator in Excel, the tidy data should be in one Excel file per table. They should not have multiple worksheets, no macros should be applied to the data, and no columns/cells should be highlighted. Alternatively share the data in a CSV or TAB-delimited text file. (Beware however that reading CSV files into Excel can sometimes lead to non-reproducible handling of date and time variables.) The code book For almost any data set, the measurements you calculate will need to be described in more detail than you can or should sneak into the spreadsheet. The code book contains this information. At minimum it should contain: Information about the variables (including units!) in the data set not contained in the tidy data Information about the summary choices you made Information about the experimental study design you used In our genomics example, the analyst would want to know what the unit of measurement for each clinical/demographic variable is (age in years, treatment by name/dose, level of diagnosis and how heterogeneous). They would also want to know how you picked the exons you used for summarizing the genomic data (UCSC/Ensembl, etc.). They would also want to know any other information about how you did the data collection/study design. For example, are these the first 20 patients that walked into the clinic? Are they 20 highly selected patients by some characteristic like age? Are they randomized to treatments? A common format for this document is a Word file. There should be a section called "Study design" that has a thorough description of how you collected the data. There is a section called "Code book" that describes each variable and its units. How to code variables When you put variables into a spreadsheet there are several main categories you will run into depending on their data type: Continuous Ordinal Categorical Missing Censored Continuous variables are anything measured on a quantitative scale that could be any fractional number. An example would be something like weight measured in kg. Ordinal data are data that have a fixed, small (< 100) number of levels but are ordered. This could be for example survey responses where the choices are: poor, fair, good. Categorical data are data where there are multiple categories, but they aren't ordered. One example would be sex: male or female. This coding is attractive because it is self-documenting. Missing data are data that are unobserved and you don't know the mechanism. You should code missing values as NA. Censored data are data where you know the missingness mechanism on some level. Common examples are a measurement being below a detection limit or a patient being lost to follow-up. They should also be coded as NA when you don't have the data. But you should also add a new column to your tidy data called, "VariableNameCensored" which should have values of TRUE if censored and FALSE if not. In the code book you should explain why those values are missing. It is absolutely critical to report to the analyst if there is a reason you know about that some of the data are missing. You should also not impute/make up/ throw away missing observations. In general, try to avoid coding categorical or ordinal variables as numbers. When you enter the value for sex in the tidy data, it should be "male" or "female". The ordinal values in the data set should be "poor", "fair", and "good" not 1, 2 ,3. This will avoid potential mixups about which direction effects go and will help identify coding errors. Always encode every piece of information about your observations using text. For example, if you are storing data in Excel and use a form of colored text or cell background formatting to indicate information about an observation ("red variable entries were observed in experiment 1.") then this information will not be exported (and will be lost!) when the data is exported as raw text. Every piece of data should be encoded as actual text that can be exported. The instruction list/script You may have heard this before, but reproducibility is a big deal in computational science. That means, when you submit your paper, the reviewers and the rest of the world should be able to exactly replicate the analyses from raw data all the way to final results. If you are trying to be efficient, you will likely perform some summarization/data analysis steps before the data can be considered tidy. The ideal thing for you to do when performing summarization is to create a computer script (in R, Python, or something else) that takes the raw data as input and produces the tidy data you are sharing as output. You can try running your script a couple of times and see if the code produces the same output. In many cases, the person who collected the data has incentive to make it tidy for a statistician to speed the process of collaboration. They may not know how to code in a scripting language. In that case, what you should provide the statistician is something called pseudocode. It should look something like: Step 1 - take the raw file, run version 3.1.2 of summarize software with parameters a=1, b=2, c=3 Step 2 - run the software separately for each sample Step 3 - take column three of outputfile.out for each sample and that is the corresponding row in the output data set You should also include information about which system (Mac/Windows/Linux) you used the software on and whether you tried it more than once to confirm it gave the same results. Ideally, you will run this by a fellow student/labmate to confirm that they can obtain the same output file you did. What you should expect from the analyst When you turn over a properly tidied data set it dramatically decreases the workload on the statistician. So hopefully they will get back to you much sooner. But most careful statisticians will check your recipe, ask questions about steps you performed, and try to confirm that they can obtain the same tidy data that you did with, at minimum, spot checks. You should then expect from the statistician: An analysis script that performs each of the analyses (not just instructions) The exact computer code they used to run the analysis All output files/figures they generated. This is the information you will use in the supplement to establish reproducibility and precision of your results. Each of the steps in the analysis should be clearly explained and you should ask questions when you don't understand what the analyst did. It is the responsibility of both the statistician and the scientist to understand the statistical analysis. You may not be able to perform the exact analyses without the statistician's code, but you should be able to explain why the statistician performed each step to a labmate/your principal investigator. Contributors Jeff Leek - Wrote the initial version. L. Collado-Torres - Fixed typos, added links. Nick Reich - Added tips on storing data as text. Nick Horton - Minor wording suggestions.

A CSV Exporter. Faster than navicat exporter

Load CSV files into Teradata table. Faster than SQLAssistant.

Modern, high-performance CSV library for .NET with source generators and Native AOT support. 2-3× faster than CsvHelper.

A fine-tuned Qwen2.5-1.5B-Instruct model for financial analysis. The assistant reads spreadsheets, PDFs, and CSVs, computes ratios, builds SWOT analyses, and generates executive summaries with 99% accuracy and ~25x faster performance than manual review.

Introduction Keeping with the spirit of the project, differences between this phase and previous will be relatively minimal besides the methods of parallelisation. Your code will read in the same movie metadata CSV files and sort the data in them with mergesort. Your code will also need to scan through the directory it is called on and sort each file in that directory and all files in all subdirectories. The major differences are that your program will instead spawn a new thread to search each directory and sort each file rather than a new process. Since all your computation will be in the same address space, you need not make separate files. The sorted output of all files will go to a single output file. Be sure that in the case of any bad input or a bad status code that your program fails gracefully, closes files, frees memory and exits threads. Under no condition should your code crash on a segmentation fault. As your code runs your threads should output similar metadata to STDOUT as the previous phase. Each thread should output the number of threads it created along with their threadIDs. Finally, you should study the differences between threads and processes for multicomputing. The metrics you should compute are detailed below. Methodology a. Command line flags Your code will read in a set of flags via the command line. They are '-c', '-d' and '-o'. Note: 'optional' and 'mandatory' below refer to your code's operation only. You must implement all three flags. Mandatory flags must be present for your code to run. Optional flags may or may not be present. If they are not, your code should take the specified default behavior. You must support all flags. '-c' indicates sorting by a column. The files read in by your code should be sorted by the column name that immediately follows '-c'. This flag is required. If it is not present, your code should print an error message, usage information and return. '-d' indicates a starting directory. The program will start at this directory name immediately following '-d' to look for CSV files to sort. This flag is optional. If this flag is not present, the default behavior of your code should be to start searching at the current directory. '-o' indicates the output directory for the sorted file. The program will store the sorted file to the directory name immediately following the '-o'. This flag is optional. If this flag is not present, the default behavior of your code should be to store the sorted file in the current directory. These flags may be defined in any order. For instance: $ ./sorter -c movie_title -d thisdir -o thatdir $ ./sorter -d thisdir -o thatdir -c movie_title $ ./sorter -d thisdir -c movie_title -o thatdir .. are all valid and would operate identically. b. File Structure CSV file structure remains unchanged. Records will be stored in CSV files in the provided directory. As mentioned above, directory structures may have multiple levels and you must find all CSV files. Your code should ignore non-CSV files and CSV files that do not have the correct format of the movie_metadata CSV (e.g. CSV files that have other random data in them). c. File Sorting Your code will be reading in and traversing entire starting directory and each contained subdirectory. Your code should open and sort each CSV file found that contains move data. Your code's output will be single CSV file outputted to a file named: AllFiles-sorted-<fieldname>.csv. Your code should create a new thread to handle each subdirectory found, and each file found. Each file thread should read in and sort that file using Mergesort. You are welcome to implement another sorting algorithm, but it must be written entirely your self. Once sorted, Instead of writing out the sorted version, your file threads should write their data in to some common data structure in the heap. Once all CSV files have been sorted your code must take the individually-sorted files and integrate all the results to create a single sorted file containing all the records read in. It would be advisable to insert the per-file data in to a globally-accessible sorted data structure. Be sure to keep this data structure propertly synchronized, however. You will need to make use of synchronization mechanisms to be sure that your updates are handled in a consistent and coherent manner. d. Metadata Your threads should write metadata to STDOUT while running. It should be in the following format and order: Initial PID: XXXXX TIDS of all child threads: AAA,BBB,CCC,DDD,EEE,FFF (... etc.) Total number of threads: ZZZZZ e. Analysis Lastly, you will use the time utility to time the execution of your program: $ time ./sorter -c movie_title -d thisdir -o thatdir real: 0mXXXs user: 0mYYYs sys: 0mZZZs You should run your Project 1 sorter on the same files/directory and record the differences. You should build a variety of different directory structures and file sizes and test your current sorter and Project 1 sorter on both. The goal is to see how multiprocessing compares in speed to multithreading. See how the two compare with: - long subdirectory chains - lots of files in a single directory - combinations of the two In particular, see how the time differs as the number of files and/or directories increase. For instance, collect timing information for both programs with a single directory and 1, 2, 4, 8, 16, 32, 64, 128, 256, 512 and 1024 files. Plot the time taken vs number of files to see if there is a definite trend for both programs. It would be a good idea to run each test multiple times and average the results. Other users, the speed of the hard drives, the OS scheduler, the difficulty in sorting the files, and any number of other factors may alter your results. It would be a very good idea to automate as much as you can. You may want to write some simple code to automatically create N files to be sorted, to run your code and collect the time results. It may sound like unnecessary extra work, but running everything manually by hand will likely take much more time than automating it. Let the machine do the boring work. Welcome to Computational Science! Results Submit your "sorter_thread.c", "sorter_thread.h" and "mergesort.c" as well as any other source files your header file references. Document your design, assumptions, difficulties you had and testing procedure. Include any test CSV files you used in your documentation. Be sure to also include a short description of how to use your code. Look at the man pages for suggestions on format and content. Do not neglect your header file. Be sure to describe the contents of it and why you needed them. Include a file called analysis.pdf that describes your findings from executing the time utility. Discuss why the results are the way they are. Be sure to answer the following questions: Is the comparison between run times a fair one? Why or why not? What are some reasons for the discrepancies of the times or for lack of discrepancies? If there are differences, is it possible to make the slower one faster? How? If there were no differences, is it possible to make one faster than the other? How? Is mergesort the right option for a multithreaded sorting program? Why or why not? Extra Credit a. (30 points) Implement the fastest (wall clock time) program. Your program will be compared against all other submitted programs in the class. The programs that are among the top 10% fastest implementations will receive the 30 points. The time that will be used is the real time reported by time. To achieve this, you may need to re-examine your data structures and algorithms to optimize your code for speed. b. (20 points) Implement a fast (wall clock time) program. If your program is not within the top 10% but is within the top 30% of timed finishers. You will receive 10 points. *In the event that all groups submit programs that run with times that are not different in statistically significant ways, we will give everyone 5 points extra credit.

Streaming csv parser inspired by binary-csv that aims to be faster than everyone else

Automated PDF invoice parser with OCR support. 90% accuracy, 10x faster than manual. Extracts financial data to CSV/JSON.

A lean FastAPI micro‑service that moves 50 GB of CSV per day—cheaper & faster than Azure Data Factory Copy.

📅 Extract dates from logs at lightning speed, 200x faster than manual methods, supporting JSON, CSV, XML, and more with ease.

I needed to save data to csv faster than with PLX-DAQ. So searched for a solution. THX to Tom Igoe for!

Fast C extension for high-performance string operations. memchr-based byte scanning up to 68x faster than Ruby stdlib. Useful for log processing, CSV parsing, streaming pipelines.

🔢 Extract numbers swiftly from JSON, YAML, CSV, TOML, INI, and ENV files at 1.5M numbers per second, 100x faster than manual searching.

Export all Zendesk users to CSV using the Incremental Export API. Handles rate limiting, connection retries, and supports unlimited users. 10x faster than standard API.

download assemblies from the mgnify api from a list of assemblies. uses the .csv output of the api which is a lot easier and faster than the json