Streaming csv parser inspired by binary-csv that aims to be faster than everyone else

A faster CSV parser in 5KB (min)

using Neural Networks (SSD) on Tensorflow. This repo documents steps and scripts used to train a hand detector using Tensorflow (Object Detection API). As with any DNN based task, the most expensive (and riskiest) part of the process has to do with finding or creating the right (annotated) dataset. I was interested mainly in detecting hands on a table (egocentric view point). I experimented first with the [Oxford Hands Dataset](http://www.robots.ox.ac.uk/~vgg/data/hands/) (the results were not good). I then tried the [Egohands Dataset](http://vision.soic.indiana.edu/projects/egohands/) which was a much better fit to my requirements. The goal of this repo/post is to demonstrate how neural networks can be applied to the (hard) problem of tracking hands (egocentric and other views). Better still, provide code that can be adapted to other uses cases. If you use this tutorial or models in your research or project, please cite [this](#citing-this-tutorial). Here is the detector in action. <img src="images/hand1.gif" width="33.3%"><img src="images/hand2.gif" width="33.3%"><img src="images/hand3.gif" width="33.3%"> Realtime detection on video stream from a webcam . <img src="images/chess1.gif" width="33.3%"><img src="images/chess2.gif" width="33.3%"><img src="images/chess3.gif" width="33.3%"> Detection on a Youtube video. Both examples above were run on a macbook pro **CPU** (i7, 2.5GHz, 16GB). Some fps numbers are: | FPS | Image Size | Device| Comments| | ------------- | ------------- | ------------- | ------------- | | 21 | 320 * 240 | Macbook pro (i7, 2.5GHz, 16GB) | Run without visualizing results| | 16 | 320 * 240 | Macbook pro (i7, 2.5GHz, 16GB) | Run while visualizing results (image above) | | 11 | 640 * 480 | Macbook pro (i7, 2.5GHz, 16GB) | Run while visualizing results (image above) | > Note: The code in this repo is written and tested with Tensorflow `1.4.0-rc0`. Using a different version may result in [some errors](https://github.com/tensorflow/models/issues/1581). You may need to [generate your own frozen model](https://pythonprogramming.net/testing-custom-object-detector-tensorflow-object-detection-api-tutorial/?completed=/training-custom-objects-tensorflow-object-detection-api-tutorial/) graph using the [model checkpoints](model-checkpoint) in the repo to fit your TF version. **Content of this document** - Motivation - Why Track/Detect hands with Neural Networks - Data preparation and network training in Tensorflow (Dataset, Import, Training) - Training the hand detection Model - Using the Detector to Detect/Track hands - Thoughts on Optimizations. > P.S if you are using or have used the models provided here, feel free to reach out on twitter ([@vykthur](https://twitter.com/vykthur)) and share your work! ## Motivation - Why Track/Detect hands with Neural Networks? There are several existing approaches to tracking hands in the computer vision domain. Incidentally, many of these approaches are rule based (e.g extracting background based on texture and boundary features, distinguishing between hands and background using color histograms and HOG classifiers,) making them not very robust. For example, these algorithms might get confused if the background is unusual or in situations where sharp changes in lighting conditions cause sharp changes in skin color or the tracked object becomes occluded.(see [here for a review](https://www.cse.unr.edu/~bebis/handposerev.pdf) paper on hand pose estimation from the HCI perspective) With sufficiently large datasets, neural networks provide opportunity to train models that perform well and address challenges of existing object tracking/detection algorithms - varied/poor lighting, noisy environments, diverse viewpoints and even occlusion. The main drawbacks to usage for real-time tracking/detection is that they can be complex, are relatively slow compared to tracking-only algorithms and it can be quite expensive to assemble a good dataset. But things are changing with advances in fast neural networks. Furthermore, this entire area of work has been made more approachable by deep learning frameworks (such as the tensorflow object detection api) that simplify the process of training a model for custom object detection. More importantly, the advent of fast neural network models like ssd, faster r-cnn, rfcn (see [here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md#coco-trained-models-coco-models) ) etc make neural networks an attractive candidate for real-time detection (and tracking) applications. Hopefully, this repo demonstrates this. > If you are not interested in the process of training the detector, you can skip straight to applying the [pretrained model I provide in detecting hands](#detecting-hands). Training a model is a multi-stage process (assembling dataset, cleaning, splitting into training/test partitions and generating an inference graph). While I lightly touch on the details of these parts, there are a few other tutorials cover training a custom object detector using the tensorflow object detection api in more detail[ see [here](https://pythonprogramming.net/training-custom-objects-tensorflow-object-detection-api-tutorial/) and [here](https://towardsdatascience.com/how-to-train-your-own-object-detector-with-tensorflows-object-detector-api-bec72ecfe1d9) ]. I recommend you walk through those if interested in training a custom object detector from scratch. ## Data preparation and network training in Tensorflow (Dataset, Import, Training) **The Egohands Dataset** The hand detector model is built using data from the [Egohands Dataset](http://vision.soic.indiana.edu/projects/egohands/) dataset. This dataset works well for several reasons. It contains high quality, pixel level annotations (>15000 ground truth labels) where hands are located across 4800 images. All images are captured from an egocentric view (Google glass) across 48 different environments (indoor, outdoor) and activities (playing cards, chess, jenga, solving puzzles etc). <img src="images/egohandstrain.jpg" width="100%"> If you will be using the Egohands dataset, you can cite them as follows: > Bambach, Sven, et al. "Lending a hand: Detecting hands and recognizing activities in complex egocentric interactions." Proceedings of the IEEE International Conference on Computer Vision. 2015. The Egohands dataset (zip file with labelled data) contains 48 folders of locations where video data was collected (100 images per folder). ``` -- LOCATION_X -- frame_1.jpg -- frame_2.jpg ... -- frame_100.jpg -- polygons.mat // contains annotations for all 100 images in current folder -- LOCATION_Y -- frame_1.jpg -- frame_2.jpg ... -- frame_100.jpg -- polygons.mat // contains annotations for all 100 images in current folder ``` **Converting data to Tensorflow Format** Some initial work needs to be done to the Egohands dataset to transform it into the format (`tfrecord`) which Tensorflow needs to train a model. This repo contains `egohands_dataset_clean.py` a script that will help you generate these csv files. - Downloads the egohands datasets - Renames all files to include their directory names to ensure each filename is unique - Splits the dataset into train (80%), test (10%) and eval (10%) folders. - Reads in `polygons.mat` for each folder, generates bounding boxes and visualizes them to ensure correctness (see image above). - Once the script is done running, you should have an images folder containing three folders - train, test and eval. Each of these folders should also contain a csv label document each - `train_labels.csv`, `test_labels.csv` that can be used to generate `tfrecords` Note: While the egohands dataset provides four separate labels for hands (own left, own right, other left, and other right), for my purpose, I am only interested in the general `hand` class and label all training data as `hand`. You can modify the data prep script to generate `tfrecords` that support 4 labels. Next: convert your dataset + csv files to tfrecords. A helpful guide on this can be found [here](https://pythonprogramming.net/creating-tfrecord-files-tensorflow-object-detection-api-tutorial/).For each folder, you should be able to generate `train.record`, `test.record` required in the training process. ## Training the hand detection Model Now that the dataset has been assembled (and your tfrecords), the next task is to train a model based on this. With neural networks, it is possible to use a process called [transfer learning](https://www.tensorflow.org/tutorials/image_retraining) to shorten the amount of time needed to train the entire model. This means we can take an existing model (that has been trained well on a related domain (here image classification) and retrain its final layer(s) to detect hands for us. Sweet!. Given that neural networks sometimes have thousands or millions of parameters that can take weeks or months to train, transfer learning helps shorten training time to possibly hours. Tensorflow does offer a few models (in the tensorflow [model zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md#coco-trained-models-coco-models)) and I chose to use the `ssd_mobilenet_v1_coco` model as my start point given it is currently (one of) the fastest models (read the SSD research [paper here](https://arxiv.org/pdf/1512.02325.pdf)). The training process can be done locally on your CPU machine which may take a while or better on a (cloud) GPU machine (which is what I did). For reference, training on my macbook pro (tensorflow compiled from source to take advantage of the mac's cpu architecture) the maximum speed I got was 5 seconds per step as opposed to the ~0.5 seconds per step I got with a GPU. For reference it would take about 12 days to run 200k steps on my mac (i7, 2.5GHz, 16GB) compared to ~5hrs on a GPU. > **Training on your own images**: Please use the [guide provided by Harrison from pythonprogramming](https://pythonprogramming.net/training-custom-objects-tensorflow-object-detection-api-tutorial/) on how to generate tfrecords given your label csv files and your images. The guide also covers how to start the training process if training locally. [see [here] (https://pythonprogramming.net/training-custom-objects-tensorflow-object-detection-api-tutorial/)]. If training in the cloud using a service like GCP, see the [guide here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/running_on_cloud.md). As the training process progresses, the expectation is that total loss (errors) gets reduced to its possible minimum (about a value of 1 or thereabout). By observing the tensorboard graphs for total loss(see image below), it should be possible to get an idea of when the training process is complete (total loss does not decrease with further iterations/steps). I ran my training job for 200k steps (took about 5 hours) and stopped at a total Loss (errors) value of 2.575.(In retrospect, I could have stopped the training at about 50k steps and gotten a similar total loss value). With tensorflow, you can also run an evaluation concurrently that assesses your model to see how well it performs on the test data. A commonly used metric for performance is mean average precision (mAP) which is single number used to summarize the area under the precision-recall curve. mAP is a measure of how well the model generates a bounding box that has at least a 50% overlap with the ground truth bounding box in our test dataset. For the hand detector trained here, the mAP value was **0.9686@0.5IOU**. mAP values range from 0-1, the higher the better. <img src="images/accuracy.jpg" width="100%"> Once training is completed, the trained inference graph (`frozen_inference_graph.pb`) is then exported (see the earlier referenced guides for how to do this) and saved in the `hand_inference_graph` folder. Now its time to do some interesting detection. ## Using the Detector to Detect/Track hands If you have not done this yet, please following the guide on installing [Tensorflow and the Tensorflow object detection api](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md). This will walk you through setting up the tensorflow framework, cloning the tensorflow github repo and a guide on - Load the `frozen_inference_graph.pb` trained on the hands dataset as well as the corresponding label map. In this repo, this is done in the `utils/detector_utils.py` script by the `load_inference_graph` method. ```python detection_graph = tf.Graph() with detection_graph.as_default(): od_graph_def = tf.GraphDef() with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid: serialized_graph = fid.read() od_graph_def.ParseFromString(serialized_graph) tf.import_graph_def(od_graph_def, name='') sess = tf.Session(graph=detection_graph) print("> ====== Hand Inference graph loaded.") ``` - Detect hands. In this repo, this is done in the `utils/detector_utils.py` script by the `detect_objects` method. ```python (boxes, scores, classes, num) = sess.run( [detection_boxes, detection_scores, detection_classes, num_detections], feed_dict={image_tensor: image_np_expanded}) ``` - Visualize detected bounding detection_boxes. In this repo, this is done in the `utils/detector_utils.py` script by the `draw_box_on_image` method. This repo contains two scripts that tie all these steps together. - detect_multi_threaded.py : A threaded implementation for reading camera video input detection and detecting. Takes a set of command line flags to set parameters such as `--display` (visualize detections), image parameters `--width` and `--height`, videe `--source` (0 for camera) etc. - detect_single_threaded.py : Same as above, but single threaded. This script works for video files by setting the video source parameter videe `--source` (path to a video file). ```cmd # load and run detection on video at path "videos/chess.mov" python detect_single_threaded.py --source videos/chess.mov ``` > Update: If you do have errors loading the frozen inference graph in this repo, feel free to generate a new graph that fits your TF version from the model-checkpoint in this repo. Use the [export_inference_graph.py](https://github.com/tensorflow/models/blob/master/research/object_detection/export_inference_graph.py) script provided in the tensorflow object detection api repo. More guidance on this [here](https://pythonprogramming.net/testing-custom-object-detector-tensorflow-object-detection-api-tutorial/?completed=/training-custom-objects-tensorflow-object-detection-api-tutorial/). ## Thoughts on Optimization. A few things that led to noticeable performance increases. - Threading: Turns out that reading images from a webcam is a heavy I/O event and if run on the main application thread can slow down the program. I implemented some good ideas from [Adrian Rosebuck](https://www.pyimagesearch.com/2017/02/06/faster-video-file-fps-with-cv2-videocapture-and-opencv/) on parrallelizing image capture across multiple worker threads. This mostly led to an FPS increase of about 5 points. - For those new to Opencv, images from the `cv2.read()` method return images in [BGR format](https://www.learnopencv.com/why-does-opencv-use-bgr-color-format/). Ensure you convert to RGB before detection (accuracy will be much reduced if you dont). ```python cv2.cvtColor(image_np, cv2.COLOR_BGR2RGB) ``` - Keeping your input image small will increase fps without any significant accuracy drop.(I used about 320 x 240 compared to the 1280 x 720 which my webcam provides). - Model Quantization. Moving from the current 32 bit to 8 bit can achieve up to 4x reduction in memory required to load and store models. One way to further speed up this model is to explore the use of [8-bit fixed point quantization](https://heartbeat.fritz.ai/8-bit-quantization-and-tensorflow-lite-speeding-up-mobile-inference-with-low-precision-a882dfcafbbd). Performance can also be increased by a clever combination of tracking algorithms with the already decent detection and this is something I am still experimenting with. Have ideas for optimizing better, please share! <img src="images/general.jpg" width="100%"> Note: The detector does reflect some limitations associated with the training set. This includes non-egocentric viewpoints, very noisy backgrounds (e.g in a sea of hands) and sometimes skin tone. There is opportunity to improve these with additional data. ## Integrating Multiple DNNs. One way to make things more interesting is to integrate our new knowledge of where "hands" are with other detectors trained to recognize other objects. Unfortunately, while our hand detector can in fact detect hands, it cannot detect other objects (a factor or how it is trained). To create a detector that classifies multiple different objects would mean a long involved process of assembling datasets for each class and a lengthy training process. > Given the above, a potential strategy is to explore structures that allow us **efficiently** interleave output form multiple pretrained models for various object classes and have them detect multiple objects on a single image. An example of this is with my primary use case where I am interested in understanding the position of objects on a table with respect to hands on same table. I am currently doing some work on a threaded application that loads multiple detectors and outputs bounding boxes on a single image. More on this soon.

Facebook marketplace bot that automatically removes and uploads listings from CSV Files. This way listings are seen by more people and that leads to selling them faster

FasterCSV is CSV, but faster, smaller, and cleaner.

W3C Strategic Highlights September 2019 This report was prepared for the September 2019 W3C Advisory Committee Meeting (W3C Member link). See the accompanying W3C Fact Sheet — September 2019. For the previous edition, see the April 2019 W3C Strategic Highlights. For future editions of this report, please consult the latest version. A Chinese translation is available. ☰ Contents Introduction Future Web Standards Meeting Industry Needs Web Payments Digital Publishing Media and Entertainment Web & Telecommunications Real-Time Communications (WebRTC) Web & Networks Automotive Web of Things Strengthening the Core of the Web HTML CSS Fonts SVG Audio Performance Web Performance WebAssembly Testing Browser Testing and Tools WebPlatform Tests Web of Data Web for All Security, Privacy, Identity Internationalization (i18n) Web Accessibility Outreach to the world W3C Developer Relations W3C Training Translations W3C Liaisons Introduction This report highlights recent work of enhancement of the existing landscape of the Web platform and innovation for the growth and strength of the Web. 33 working groups and a dozen interest groups enable W3C to pursue its mission through the creation of Web standards, guidelines, and supporting materials. We track the tremendous work done across the Consortium through homogeneous work-spaces in Github which enables better monitoring and management. We are in the middle of a period where we are chartering numerous working groups which demonstrate the rapid degree of change for the Web platform: After 4 years, we are nearly ready to publish a Payment Request API Proposed Recommendation and we need to soon charter follow-on work. In the last year we chartered the Web Payment Security Interest Group. In the last year we chartered the Web Media Working Group with 7 specifications for next generation Media support on the Web. We have Accessibility Guidelines under W3C Member review which includes Silver, a new approach. We have just launched the Decentralized Identifier Working Group which has tremendous potential because Decentralized Identifier (DID) is an identifier that is globally unique, resolveable with high availability, and cryptographically verifiable. We have Privacy IG (PING) under W3C Member review which strengthens our focus on the tradeoff between privacy and function. We have a new CSS charter under W3C Member review which maps the group's work for the next three years. In this period, W3C and the WHATWG have succesfully completed the negotiation of a Memorandum of Understanding rooted in the mutual belief that that having two distinct specifications claiming to be normative is generally harmful for the Web community. The MOU, signed last May, describes how the two organizations are to collaborate on the development of a single authoritative version of the HTML and DOM specifications. W3C subsequently rechartered the HTML Working Group to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and for the production of W3C Recommendations from WHATWG Review Drafts. As the Web evolves continuously, some groups are looking for ways for specifications to do so as well. So-called "evergreen recommendations" or "living standards" aim to track continuous development (and maintenance) of features, on a feature-by-feature basis, while getting review and patent commitments. We see the maturation and further development of an incredible number of new technologies coming to the Web. Continued progress in many areas demonstrates the vitality of the W3C and the Web community, as the rest of the report illustrates. Future Web Standards W3C has a variety of mechanisms for listening to what the community thinks could become good future Web standards. These include discussions with the Membership, discussions with other standards bodies, the activities of thousands of participants in over 300 community groups, and W3C Workshops. There are lots of good ideas. The W3C strategy team has been identifying promising topics and invites public participation. Future, recent and under consideration Workshops include: Inclusive XR (5-6 November 2019, Seattle, WA, USA) to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive, including to people with disabilities; W3C Workshop on Data Models for Transportation (12-13 September 2019, Palo Alto, CA, USA) W3C Workshop on Web Games (27-28 June 2019, Redmond, WA, USA), view report Second W3C Workshop on the Web of Things (3-5 June 2019, Munich, Germany) W3C Workshop on Web Standardization for Graph Data; Creating Bridges: RDF, Property Graph and SQL (4-6 March 2019, Berlin, Germany), view report Web & Machine Learning. The Strategy Funnel documents the staff's exploration of potential new work at various phases: Exploration and Investigation, Incubation and Evaluation, and eventually to the chartering of a new standards group. The Funnel view is a GitHub Project where new area are issues represented by “cards” which move through the columns, usually from left to right. Most cards start in Exploration and move towards Chartering, or move out of the funnel. Public input is welcome at any stage but particularly once Incubation has begun. This helps W3C identify work that is sufficiently incubated to warrant standardization, to review the ecosystem around the work and indicate interest in participating in its standardization, and then to draft a charter that reflects an appropriate scope. Ongoing feedback can speed up the overall standardization process. Since the previous highlights document, W3C has chartered a number of groups, and started discussion on many more: Newly Chartered or Rechartered Web Application Security WG (03-Apr) Web Payment Security IG (17-Apr) Patent and Standards IG (24-Apr) Web Applications WG (14-May) Web & Networks IG (16-May) Media WG (23-May) Media and Entertainment IG (06-Jun) HTML WG (06-Jun) Decentralized Identifier WG (05-Sep) Extended Privacy IG (PING) (30-Sep) Verifiable Claims WG (30-Sep) Service Workers WG (31-Dec) Dataset Exchange WG (31-Dec) Web of Things Working Group (31-Dec) Web Audio Working Group (31-Dec) Proposed charters / Advance Notice Accessibility Guidelines WG Privacy IG (PING) RDF Literal Direction WG Timed Text WG CSS WG Web Authentication WG Closed Internationalization Tag Set IG Meeting Industry Needs Web Payments All Web Payments specifications W3C's payments standards enable a streamlined checkout experience, enabling a consistent user experience across the Web with lower front end development costs for merchants. Users can store and reuse information and more quickly and accurately complete online transactions. The Web Payments Working Group has republished Payment Request API as a Candidate Recommendation, aiming to publish a Proposed Recommendation in the Fall 2019, and is discussing use cases and features for Payment Request after publication of the 1.0 Recommendation. Browser vendors have been finalizing implementation of features added in the past year (view the implementation report). As work continues on the Payment Handler API and its implementation (currently in Chrome and Edge Canary), one focus in 2019 is to increase adoption in other browsers. Recently, Mastercard demonstrated the use of Payment Request API to carry out EMVCo's Secure Remote Commerce (SRC) protocol whose payment method definition is being developed with active participation by Visa, Mastercard, American Express, and Discover. Payment method availability is a key factor in merchant considerations about adopting Payment Request API. The ability to get uniform adoption of a new payment method such as Secure Remote Commerce (SRC) also depends on the availability of the Payment Handler API in browsers, or of proprietary alternatives. Web Monetization, which the Web Payments Working Group will discuss again at its face-to-face meeting in September, can be used to enable micropayments as an alternative revenue stream to advertising. Since the beginning of 2019, Amazon, Brave Software, JCB, Certus Cybersecurity Solutions and Netflix have joined the Web Payments Working Group. In April, W3C launched the Web Payment Security Group to enable W3C, EMVCo, and the FIDO Alliance to collaborate on a vision for Web payment security and interoperability. Participants will define areas of collaboration and identify gaps between existing technical specifications in order to increase compatibility among different technologies, such as: How do SRC, FIDO, and Payment Request relate? The Payment Services Directive 2 (PSD2) regulations in Europe are scheduled to take effect in September 2019. What is the role of EMVCo, W3C, and FIDO technologies, and what is the current state of readiness for the deadline? How can we improve privacy on the Web at the same time as we meet industry requirements regarding user identity? Digital Publishing All Digital Publishing specifications, Publication milestones The Web is the universal publishing platform. Publishing is increasingly impacted by the Web, and the Web increasingly impacts Publishing. Topic of particular interest to Publishing@W3C include typography and layout, accessibility, usability, portability, distribution, archiving, offline access, print on demand, and reliable cross referencing. And the diverse publishing community represented in the groups consist of the traditional "trade" publishers, ebook reading system manufacturers, but also publishers of audio book, scholarly journals or educational materials, library scientists or browser developers. The Publishing Working Group currently concentrates on Audiobooks which lack a comprehensive standard, thus incurring extra costs and time to publish in this booming market. Active development is ongoing on the future standard: Publication Manifest Audiobook profile for Web Publications Lightweight Packaging Format The BD Comics Manga Community Group, the Synchronized Multimedia for Publications Community Group, the Publishing Community Group and a future group on archival, are companions to the working group where specific work is developed and incubated. The Publishing Community Group is a recently launched incubation channel for Publishing@W3C. The goal of the group is to propose, document, and prototype features broadly related to: publications on the Web reading modes and systems and the user experience of publications The EPUB 3 Community Group has successfully completed the revision of EPUB 3.2. The Publishing Business Group fosters ongoing participation by members of the publishing industry and the overall ecosystem in the development of Web infrastructure to better support the needs of the industry. The Business Group serves as an additional conduit to the Publishing Working Group and several Community Groups for feedback between the publishing ecosystem and W3C. The Publishing BG has played a vital role in fostering and advancing the adoption and continued development of EPUB 3. In particular the BG provided critical support to the update of EPUBCheck to validate EPUB content to the new EPUB 3.2 specification. This resulted in the development, in conjunction with the EPUB3 Community Group, of a new generation of EPUBCheck, i.e., EPUBCheck 4.2 production-ready release. Media and Entertainment All Media specifications The Media and Entertainment vertical tracks media-related topics and features that create immersive experiences for end users. HTML5 brought standard audio and video elements to the Web. Standardization activities since then have aimed at turning the Web into a professional platform fully suitable for the delivery of media content and associated materials, enabling missing features to stream video content on the Web such as adaptive streaming and content protection. Together with Microsoft, Comcast, Netflix and Google, W3C received an Technology & Engineering Emmy Award in April 2019 for standardization of a full TV experience on the Web. Current goals are to: Reinforce core media technologies: Creation of the Media Working Group, to develop media-related specifications incubated in the WICG (e.g. Media Capabilities, Picture-in-picture, Media Session) and maintain maintain/evolve Media Source Extensions (MSE) and Encrypted Media Extensions (EME). Improve support for Media Timed Events: data cues incubation. Enhance color support (HDR, wide gamut), in scope of the CSS WG and in the Color on the Web CG. Reduce fragmentation: Continue annual releases of a common and testable baseline media devices, in scope of the Web Media APIs CG and in collaboration with the CTA WAVE Project. Maintain the Road-map of Media Technologies for the Web which highlights Web technologies that can be used to build media applications and services, as well as known gaps to enable additional use cases. Create the future: Discuss perspectives for Media and Entertainment for the Web. Bring the power of GPUs to the Web (graphics, machine learning, heavy processing), under incubation in the GPU for the Web CG. Transition to a Working Group is under discussion. Determine next steps after the successful W3C Workshop on Web Games of June 2019. View the report. Timed Text The Timed Text Working Group develops and maintains formats used for the representation of text synchronized with other timed media, like audio and video, and notably works on TTML, profiles of TTML, and WebVTT. Recent progress includes: A robust WebVTT implementation report poises the specification for publication as a proposed recommendation. Discussions around re-chartering, notably to add a TTML Profile for Audio Description deliverable to the scope of the group, and clarify that rendering of captions within XR content is also in scope. Immersive Web Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The Immersive Web Working Group has been stabilizing the WebXR Device API while the companion Immersive Web Community Group incubates the next series of features identified as key for the future of the Immersive Web. W3C plans a workshop focused on the needs and benefits at the intersection of VR & Accessibility (Inclusive XR), on 5-6 November 2019 in Seattle, WA, USA, to explore existing and future approaches on making Virtual and Augmented Reality experiences more inclusive. Web & Telecommunications The Web is the Open Platform for Mobile. Telecommunication service providers and network equipment providers have long been critical actors in the deployment of Web technologies. As the Web platform matures, it brings richer and richer capabilities to extend existing services to new users and devices, and propose new and innovative services. Real-Time Communications (WebRTC) All Real-Time Communications specifications WebRTC has reshaped the whole communication landscape by making any connected device a potential communication end-point, bringing audio and video communications anywhere, on any network, vastly expanding the ability of operators to reach their customers. WebRTC serves as the corner-stone of many online communication and collaboration services. The WebRTC Working Group aims to bringing WebRTC 1.0 (and companion specification Media Capture and Streams) to Recommendation by the end of 2019. Intense efforts are focused on testing (supported by a dedicated hackathon at IETF 104) and interoperability. The group is considering pushing features that have not gotten enough traction to separate modules or to a later minor revision of the spec. Beyond WebRTC 1.0, the WebRTC Working Group will focus its efforts on WebRTC NV which the group has started documenting by identifying use cases. Web & Networks Recently launched, in the wake of the May 2018 Web5G workshop, the Web & Networks Interest Group is chaired by representatives from AT&T, China Mobile and Intel, with a goal to explore solutions for web applications to achieve better performance and resource allocation, both on the device and network. The group's first efforts are around use cases, privacy & security requirements and liaisons. Automotive All Automotive specifications To create a rich application ecosystem for vehicles and other devices allowed to connect to the vehicle, the W3C Automotive Working Group is delivering a service specification to expose all common vehicle signals (engine temperature, fuel/charge level, range, tire pressure, speed, etc.) The Vehicle Information Service Specification (VISS), which is a Candidate Recommendation, is seeing more implementations across the industry. It provides the access method to a common data model for all the vehicle signals –presently encapsulating a thousand or so different data elements– and will be growing to accommodate the advances in automotive such as autonomous and driver assist technologies and electrification. The group is already working on a successor to VISS, leveraging the underlying data model and the VIWI submission from Volkswagen, for a more robust means of accessing vehicle signals information and the same paradigm for other automotive needs including location-based services, media, notifications and caching content. The Automotive and Web Platform Business Group acts as an incubator for prospective standards work. One of its task forces is using W3C VISS in performing data sampling and off-boarding the information to the cloud. Access to the wealth of information that W3C's auto signals standard exposes is of interest to regulators, urban planners, insurance companies, auto manufacturers, fleet managers and owners, service providers and others. In addition to components needed for data sampling and edge computing, capturing user and owner consent, information collection methods and handling of data are in scope. The upcoming W3C Workshop on Data Models for Transportation (September 2019) is expected to focus on the need of additional ontologies around transportation space. Web of Things All Web of Things specifications W3C's Web of Things work is designed to bridge disparate technology stacks to allow devices to work together and achieve scale, thus enabling the potential of the Internet of Things by eliminating fragmentation and fostering interoperability. Thing descriptions expressed in JSON-LD cover the behavior, interaction affordances, data schema, security configuration, and protocol bindings. The Web of Things complements existing IoT ecosystems to reduce the cost and risk for suppliers and consumers of applications that create value by combining multiple devices and information services. There are many sectors that will benefit, e.g. smart homes, smart cities, smart industry, smart agriculture, smart healthcare and many more. The Web of Things Working Group is finishing the initial Web of Things standards, with support from the Web of Things Interest Group: Web of Things Architecture Thing Descriptions Strengthening the Core of the Web HTML The HTML Working Group was chartered early June to assist the W3C community in raising issues and proposing solutions for the HTML and DOM specifications, and to produce W3C Recommendations from WHATWG Review Drafts. A few days before, W3C and the WHATWG signed a Memorandum of Understanding outlining the agreement to collaborate on the development of a single version of the HTML and DOM specifications. Issues and proposed solutions for HTML and DOM done via the newly rechartered HTML Working Group in the WHATWG repositories The HTML Working Group is targetting November 2019 to bring HTML and DOM to Candidate Recommendations. CSS All CSS specifications CSS is a critical part of the Open Web Platform. The CSS Working Group gathers requirements from two large groups of CSS users: the publishing industry and application developers. Within W3C, those groups are exemplified by the Publishing groups and the Web Platform Working Group. The former requires things like better pagination support and advanced font handling, the latter needs intelligent (and fast!) scrolling and animations. What we know as CSS is actually a collection of almost a hundred specifications, referred to as ‘modules’. The current state of CSS is defined by a snapshot, updated once a year. The group also publishes an index defining every term defined by CSS specifications. Fonts All Fonts specifications The Web Fonts Working Group develops specifications that allow the interoperable deployment of downloadable fonts on the Web, with a focus on Progressive Font Enrichment as well as maintenance of WOFF Recommendations. Recent and ongoing work includes: Early API experiments by Adobe and Monotype have demonstrated the feasibility of a font enrichment API, where a server delivers a font with minimal glyph repertoire and the client can query the full repertoire and request additional subsets on-the-fly. In other experiments, the Brotli compression used in WOFF 2 was extended to support shared dictionaries and patch update. Metrics to quantify improvement are a current hot discussion topic. The group will meet at ATypi 2019 in Japan, to gather requirements from the international typography community. The group will first produce a report summarizing the strengths and weaknesses of each prototype solution by Q2 2020. SVG All SVG specifications SVG is an important and widely-used part of the Open Web Platform. The SVG Working Group focuses on aligning the SVG 2.0 specification with browser implementations, having split the specification into a currently-implemented 2.0 and a forward-looking 2.1. Current activity is on stabilization, increased integration with the Open Web Platform, and test coverage analysis. The Working Group was rechartered in March 2019. A new work item concerns native (non-Web-browser) uses of SVG as a non-interactive, vector graphics format. Audio The Web Audio Working Group was extended to finish its work on the Web Audio API, expecting to publish it as a Recommendation by year end. The specification enables synthesizing audio in the browser. Audio operations are performed with audio nodes, which are linked together to form a modular audio routing graph. Multiple sources — with different types of channel layout — are supported. This modular design provides the flexibility to create complex audio functions with dynamic effects. The first version of Web Audio API is now feature complete and is implemented in all modern browsers. Work has started on the next version, and new features are being incubated in the Audio Community Group. Performance Web Performance All Web Performance specifications There are currently 18 specifications in development in the Web Performance Working Group aiming to provide methods to observe and improve aspects of application performance of user agent features and APIs. The W3C team is looking at related work incubated in the W3C GPU for the Web (WebGPU) Community Group which is poised to transition to a W3C Working Group. A preliminary draft charter is available. WebAssembly All WebAssembly specifications WebAssembly improves Web performance and power by being a virtual machine and execution environment enabling loaded pages to run native (compiled) code. It is deployed in Firefox, Edge, Safari and Chrome. The specification will soon reach Candidate Recommendation. WebAssembly enables near-native performance, optimized load time, and perhaps most importantly, a compilation target for existing code bases. While it has a small number of native types, much of the performance increase relative to Javascript derives from its use of consistent typing. WebAssembly leverages decades of optimization for compiled languages and the byte code is optimized for compactness and streaming (the web page starts executing while the rest of the code downloads). Network and API access all occurs through accompanying Javascript libraries -- the security model is identical to that of Javascript. Requirements gathering and language development occur in the Community Group while the Working Group manages test development, community review and progression of specifications on the Recommendation Track. Testing Browser testing plays a critical role in the growth of the Web by: Improving the reliability of Web technology definitions; Improving the quality of implementations of these technologies by helping vendors to detect bugs in their products; Improving the data available to Web developers on known bugs and deficiencies of Web technologies by publishing results of these tests. Browser Testing and Tools The Browser Testing and Tools Working Group is developing WebDriver version 2, having published last year the W3C Recommendation of WebDriver. WebDriver acts as a remote control interface that enables introspection and control of user agents, provides a platform- and language-neutral wire protocol as a way for out-of-process programs to remotely instruct the behavior of Web, and emulates the actions of a real person using the browser. WebPlatform Tests The WebPlatform Tests project now provides a mechanism which allows to fully automate tests that previously needed to be run manually: TestDriver. TestDriver enables sending trusted key and mouse events, sending complex series of trusted pointer and key interactions for things like in-content drag-and-drop or pinch zoom, and even file upload. Since 2014 W3C began work on this coordinated open-source effort to build a cross-browser test suite for the Web Platform, which WHATWG, and all major browsers adopted. Web of Data All Data specifications There have been several great success stories around the standardization of data on the web over the past year. Verifiable Claims seems to have significant uptake. It is also significant that the Distributed Identifier WG charter has received numerous favorable reviews, and was just recently launched. JSON-LD has been a major success with the large deployment on Web sites via schema.org. JSON-LD 1.1 completed technical work, about to transition to CR More than 25% of websites today include schema.org data in JSON-LD The Web of Things description is in CR since May, making use of JSON-LD Verifiable Credentials data model is in CR since July, also making use of JSON-LD Continued strong interest in decentralized identifiers Engagement from the TAG with reframing core documents, such as Ethical Web Principles, to include data on the web within their scope Data is increasingly important for all organizations, especially with the rise of IoT and Big Data. W3C has a mature and extensive suite of standards relating to data that were developed over two decades of experience, with plans for further work on making it easier for developers to work with graph data and knowledge graphs. Linked Data is about the use of URIs as names for things, the ability to dereference these URIs to get further information and to include links to other data. There are ever-increasing sources of open Linked Data on the Web, as well as data services that are restricted to the suppliers and consumers of those services. The digital transformation of industry is seeking to exploit advanced digital technologies. This will facilitate businesses to integrate horizontally along the supply and value chains, and vertically from the factory floor to the office floor. W3C is seeking to make it easier to support enterprise-wide data management and governance, reflecting the strategic importance of data to modern businesses. Traditional approaches to data have focused on tabular databases (SQL/RDBMS), Comma Separated Value (CSV) files, and data embedded in PDF documents and spreadsheets. We're now in midst of a major shift to graph data with nodes and labeled directed links between them. Graph data is: Faster than using SQL and associated JOIN operations More favorable to integrating data from heterogeneous sources Better suited to situations where the data model is evolving In the wake of the recent W3C Workshop on Graph Data we are in the process of launching a Graph Standardization Business Group to provide a business perspective with use cases and requirements, to coordinate technical standards work and liaisons with external organizations. Web for All Security, Privacy, Identity All Security specifications, all Privacy specifications Authentication on the Web As the WebAuthn Level 1 W3C Recommendation published last March is seeing wide implementation and adoption of strong cryptographic authentication, work is proceeding on Level 2. The open standard Web API gives native authentication technology built into native platforms, browsers, operating systems (including mobile) and hardware, offering protection against hacking, credential theft, phishing attacks, thus aiming to end the era of passwords as a security construct. You may read more in our March press release. Privacy An increasing number of W3C specifications are benefitting from Privacy and Security review; there are security and privacy aspects to every specification. Early review is essential. Working with the TAG, the Privacy Interest Group has updated the Self-Review Questionnaire: Security and Privacy. Other recent work of the group includes public blogging further to the exploration of anti-patterns in standards and permission prompts. Security The Web Application Security Working Group adopted Feature Policy, aiming to allow developers to selectively enable, disable, or modify the behavior of some of these browser features and APIs within their application; and Fetch Metadata, aiming to provide servers with enough information to make a priori decisions about whether or not to service a request based on the way it was made, and the context in which it will be used. The Web Payment Security Interest Group, launched last April, convenes members from W3C, EMVCo, and the FIDO Alliance to discuss cooperative work to enhance the security and interoperability of Web payments (read more about payments). Internationalization (i18n) All Internationalization specifications, educational articles related to Internationalization, spec developers checklist Only a quarter or so current Web users use English online and that proportion will continue to decrease as the Web reaches more and more communities of limited English proficiency. If the Web is to live up to the "World Wide" portion of its name, and for the Web to truly work for stakeholders all around the world engaging with content in various languages, it must support the needs of worldwide users as they engage with content in the various languages. The growth of epublishing also brings requirements for new features and improved typography on the Web. It is important to ensure the needs of local communities are captured. The W3C Internationalization Initiative was set up to increase in-house resources dedicated to accelerating progress in making the World Wide Web "worldwide" by gathering user requirements, supporting developers, and education & outreach. For an overview of current projects see the i18n radar. W3C's Internationalization efforts progressed on a number of fronts recently: Requirements: New African and European language groups will work on the gap analysis, errata and layout requirements. Gap analysis: Japanese, Devanagari, Bengali, Tamil, Lao, Khmer, Javanese, and Ethiopic updated in the gap-analysis documents. Layout requirements document: notable progress tracked in the Southeast Asian Task Force while work continues on Chinese layout requirements. Developer support: Spec reviews: the i18n WG continues active review of specifications of the WHATWG and other W3C Working Groups. Short review checklist: easy way to begin a self-review to help spec developers understand what aspects of their spec are likely to need attention for internationalization, and points them to more detailed checklists for the relevant topics. It also helps those reviewing specs for i18n issues. Strings on the Web: Language and Direction Metadata lays out issues and discusses potential solutions for passing information about language and direction with strings in JSON or other data formats. The document was rewritten for clarity, and expanded. The group is collaborating with the JSON-LD and Web Publishing groups to develop a plan for updating RDF, JSON-LD and related specifications to handle metadata for base direction of text (bidi). User-friendly test format: a new format was developed for Internationalization Test Suite tests, which displays helpful information about how the test works. This particularly useful because those tests are pointed to by educational materials and gap-analysis documents. Web Platform Tests: a large number of tests in the i18n test suite have been ported to the WPT repository, including: css-counter-styles, css-ruby, css-syntax, css-test, css-text-decor, css-writing-modes, and css-pseudo. Education & outreach: (for all educational materials, see the HTML & CSS Authoring Techniques) Web Accessibility All Accessibility specifications, WAI resources The Web Accessibility Initiative supports W3C's Web for All mission. Recent achievements include: Education and training: Inaccessibility of CAPTCHA updated to bring our analysis and recommendations up to date with CAPTCHA practice today, concluding two years of extensive work and invaluable input from the public (read more on the W3C Blog Learn why your web content and applications should be accessible. The Education and Outreach Working Group has completed revision and updating of the Business Case for Digital Accessibility. Accessibility guidelines: The Accessibility Guidelines Working Group has continued to update WCAG Techniques and Understanding WCAG 2.1; and published a Candidate Recommendation of Accessibility Conformance Testing Rules Format 1.0 to improve inter-rater reliability when evaluating conformance of web content to WCAG An updated charter is being developed to host work on "Silver", the next generation accessibility guidelines (WCAG 2.2) There are accessibility aspects to most specifications. Check your work with the FAST checklist. Outreach to the world W3C Developer Relations To foster the excellent feedback loop between Web Standards development and Web developers, and to grow participation from that diverse community, recent W3C Developer Relations activities include: @w3cdevs tracks the enormous amount of work happening across W3C W3C Track during the Web Conference 2019 in San Francisco Tech videos: W3C published the 2019 Web Games Workshop videos The 16 September 2019 Developer Meetup in Fukuoka, Japan, is open to all and will combine a set of technical demos prepared by W3C groups, and a series of talks on a selected set of W3C technologies and projects W3C is involved with Mozilla, Google, Samsung, Microsoft and Bocoup in the organization of ViewSource 2019 in Amsterdam (read more on the W3C Blog) W3C Training In partnership with EdX, W3C's MOOC training program, W3Cx offers a complete "Front-End Web Developer" (FEWD) professional certificate program that consists of a suite of five courses on the foundational languages that power the Web: HTML5, CSS and JavaScript. We count nearly 900K students from all over the world. Translations Many Web users rely on translations of documents developed at W3C whose official language is English. W3C is extremely grateful to the continuous efforts of its community in ensuring our various deliverables in general, and in our specifications in particular, are made available in other languages, for free, ensuring their exposure to a much more diverse set of readers. Last Spring we developed a more robust system, a new listing of translations of W3C specifications and updated the instructions on how to contribute to our translation efforts. W3C Liaisons Liaisons and coordination with numerous organizations and Standards Development Organizations (SDOs) is crucial for W3C to: make sure standards are interoperable coordinate our respective agenda in Internet governance: W3C participates in ICANN, GIPO, IGF, the I* organizations (ICANN, IETF, ISOC, IAB). ensure at the government liaison level that our standards work is officially recognized when important to our membership so that products based on them (often done by our members) are part of procurement orders. W3C has ARO/PAS status with ISO. W3C participates in the EU MSP and Rolling Plan on Standardization ensure the global set of Web and Internet standards form a compatible stack of technologies, at the technical and policy level (patent regime, fragmentation, use in policy making) promote Standards adoption equally by the industry, the public sector, and the public at large Coralie Mercier, Editor, W3C Marketing & Communications $Id: Overview.html,v 1.60 2019/10/15 12:05:52 coralie Exp $ Copyright © 2019 W3C ® (MIT, ERCIM, Keio, Beihang) Usage policies apply.

Faster CSV for Python

A faster and more eloquent way of importing and exporting Excel and CSV in Laravel with the speed of Spout.

⚡ Production-grade RAG chunking engine powered by Rust. Process GBs of CSV, PDF, JSON, JSONL, DOCX, XLSX, URLs, ETC., in seconds with O(1) memory. 40x faster than LangChain.

Fastest CSV class for MRI Ruby and JRuby. Faster than faster_csv and fasterer-csv.

json to csv conversion. about 10x faster than jq.

High-performance Excel and CSV file generation powered by Rust (~7-9x faster)

SIMD-accelerated CSV parser for Ruby - 5-6x faster than CSV stdlib

A faster implementation fo a CSV parser than FasterCSV

############################################################ # +------------------------------------------------------+ # # | Notes | # # +------------------------------------------------------+ # ############################################################ # If you want to use special characters in this document, such as accented letters, you MUST save the file as UTF-8, not ANSI. # If you receive an error when Essentials loads, ensure that: # - No tabs are present: YAML only allows spaces # - Indents are correct: YAML hierarchy is based entirely on indentation # - You have "escaped" all apostrophes in your text: If you want to write "don't", for example, write "don''t" instead (note the doubled apostrophe) # - Text with symbols is enclosed in single or double quotation marks # If you have problems join the Essentials help support channel: http://tiny.cc/EssentialsChat ############################################################ # +------------------------------------------------------+ # # | Essentials (Global) | # # +------------------------------------------------------+ # ############################################################ # A color code between 0-9 or a-f. Set to 'none' to disable. ops-name-color: 'none' # The character(s) to prefix all nicknames, so that you know they are not true usernames. nickname-prefix: '~' # Disable this if you have any other plugin, that modifies the displayname of a user. change-displayname: true # When this option is enabled, the (tab) player list will be updated with the displayname. # The value of change-displayname (above) has to be true. #change-playerlist: true # When essentialschat.jar isnt used, force essentials to add the prefix and suffix from permission plugins to displayname # This setting is ignored if essentialschat.jar is used, and defaults to 'true' # The value of change-displayname (above) has to be true. # Do not edit this setting unless you know what you are doing! #add-prefix-suffix: false # The delay, in seconds, required between /home, /tp, etc. teleport-cooldown: 5 # The delay, in seconds, before a user actually teleports. If the user moves or gets attacked in this timeframe, the teleport never occurs. teleport-delay: 5 # The delay, in seconds, a player can't be attacked by other players after they have been teleported by a command # This will also prevent the player attacking other players teleport-invulnerability: 4 # The delay, in seconds, required between /heal attempts heal-cooldown: 60 # What to prevent from /i /give # e.g item-spawn-blacklist: 46,11,10 item-spawn-blacklist: # Set this to true if you want permission based item spawn rules # Note: The blacklist above will be ignored then. # Permissions: # - essentials.itemspawn.item-all # - essentials.itemspawn.item-[itemname] # - essentials.itemspawn.item-[itemid] # - essentials.give.item-all # - essentials.give.item-[itemname] # - essentials.give.item-[itemid] # For more information, visit http://wiki.ess3.net/wiki/Command_Reference/ICheat#Item.2FGive permission-based-item-spawn: false # Mob limit on the /spawnmob command per execution spawnmob-limit: 10 # Shall we notify users when using /lightning warn-on-smite: true # motd and rules are now configured in the files motd.txt and rules.txt # When a command conflicts with another plugin, by default, Essentials will try to force the OTHER plugin to take priority. # Commands in this list, will tell Essentials to 'not give up' the command to other plugins. # In this state, which plugin 'wins' appears to be almost random. # # If you have two plugin with the same command and you wish to force Essentials to take over, you need an alias. # To force essentials to take 'god' alias 'god' to 'egod'. # See http://wiki.bukkit.org/Bukkit.yml#aliases for more information overridden-commands: # - god # Disabled commands will be completely unavailable on the server. # Disabling commands here will have no effect on command conflicts. disabled-commands: # - nick # These commands will be shown to players with socialSpy enabled # You can add commands from other plugins you may want to track or # remove commands that are used for something you dont want to spy on socialspy-commands: - msg - w - r - mail - m - t - whisper - emsg - tell - er - reply - ereply - email - action - describe - eme - eaction - edescribe - etell - ewhisper - pm # If you do not wish to use a permission system, you can define a list of 'player perms' below. # This list has no effect if you are using a supported permissions system. # If you are using an unsupported permissions system simply delete this section. # Whitelist the commands and permissions you wish to give players by default (everything else is op only). # These are the permissions without the "essentials." part. player-commands: - afk - afk.auto - back - back.ondeath - balance - balance.others - balancetop - build - chat.color - chat.format - chat.shout - chat.question - clearinventory - compass - depth - delhome - getpos - geoip.show - help - helpop - home - home.others - ignore - info - itemdb - kit - kits.tools - list - mail - mail.send - me - motd - msg - msg.color - nick - near - pay - ping - protect - r - rules - realname - seen - sell - sethome - setxmpp - signs.create.protection - signs.create.trade - signs.break.protection - signs.break.trade - signs.use.balance - signs.use.buy - signs.use.disposal - signs.use.enchant - signs.use.free - signs.use.gamemode - signs.use.heal - signs.use.info - signs.use.kit - signs.use.mail - signs.use.protection - signs.use.repair - signs.use.sell - signs.use.time - signs.use.trade - signs.use.warp - signs.use.weather - spawn - suicide - time - tpa - tpaccept - tpahere - tpdeny - warp - warp.list - world - worth - xmpp # Note: All items MUST be followed by a quantity! # All kit names should be lower case, and will be treated as lower in permissions/costs. # Syntax: - itemID[:DataValue/Durability] Amount [Enchantment:Level].. [itemmeta:value]... # For Item meta information visit http://wiki.ess3.net/wiki/Item_Meta # 'delay' refers to the cooldown between how often you can use each kit, measured in seconds. # For more information, visit http://wiki.ess3.net/wiki/Kits kits: tools: delay: 10 items: - 272 1 - 273 1 - 274 1 - 275 1 dtools: delay: 600 items: - 278 1 efficiency:1 durability:1 fortune:1 name:&4Gigadrill lore:The_drill_that_&npierces|the_heavens - 277 1 digspeed:3 name:Dwarf lore:Diggy|Diggy|Hole - 298 1 color:255,255,255 name:Top_Hat lore:Good_day,_Good_day - 279:780 1 notch: delay: 6000 items: - 397:3 1 player:Notch color: delay: 6000 items: - 387 1 title:&4Book_&9o_&6Colors author:KHobbits lore:Ingame_color_codes book:Colors firework: delay: 6000 items: - 401 1 name:Angry_Creeper color:red fade:green type:creeper power:1 - 401 1 name:StarryNight color:yellow,orange fade:blue type:star effect:trail,twinkle power:1 - 401 2 name:SolarWind color:yellow,orange fade:red shape:large effect:twinkle color:yellow,orange fade:red shape:ball effect:trail color:red,purple fade:pink shape:star effect:trail power:1 # Essentials Sign Control # See http://wiki.ess3.net/wiki/Sign_Tutorial for instructions on how to use these. # To enable signs, remove # symbol. To disable all signs, comment/remove each sign. # Essentials Colored sign support will be enabled when any sign types are enabled. # Color is not an actual sign, it's for enabling using color codes on signs, when the correct permissions are given. enabledSigns: #- color #- balance #- buy #- sell #- trade #- free #- disposal #- warp #- kit #- mail #- enchant #- gamemode #- heal #- info #- spawnmob #- repair #- time #- weather # How many times per second can Essentials signs be interacted with per player. # Values should be between 1-20, 20 being virtually no lag protection. # Lower numbers will reduce the possibility of lag, but may annoy players. sign-use-per-second: 4 # Backup runs a batch/bash command while saving is disabled backup: # Interval in minutes interval: 30 # Unless you add a valid backup command or script here, this feature will be useless. # Use 'save-all' to simply force regular world saving without backup. #command: 'rdiff-backup World1 backups/World1' # Set this true to enable permission per warp. per-warp-permission: false # Sort output of /list command by groups sort-list-by-groups: false # More output to the console debug: false # Set the locale for all messages # If you don't set this, the default locale of the server will be used. # For example, to set language to English, set locale to en, to use the file "messages_en.properties" # Don't forget to remove the # in front of the line # For more information, visit http://wiki.ess3.net/wiki/Locale #locale: en # Turn off god mode when people exit remove-god-on-disconnect: false # Auto-AFK # After this timeout in seconds, the user will be set as afk. # Set to -1 for no timeout. auto-afk: 300 # Auto-AFK Kick # After this timeout in seconds, the user will be kicked from the server. # Set to -1 for no timeout. auto-afk-kick: -1 # Set this to true, if you want to freeze the player, if he is afk. # Other players or monsters can't push him out of afk mode then. # This will also enable temporary god mode for the afk player. # The player has to use the command /afk to leave the afk mode. freeze-afk-players: false # When the player is afk, should he be able to pickup items? # Enable this, when you don't want people idling in mob traps. disable-item-pickup-while-afk: false # This setting controls if a player is marked as active on interaction. # When this setting is false, you will need to manually un-AFK using the /afk command. cancel-afk-on-interact: true # Should we automatically remove afk status when the player moves? # Player will be removed from AFK on chat/command regardless of this setting. # Disable this to reduce server lag. cancel-afk-on-move: true # You can disable the death messages of Minecraft here death-messages: true # Add worlds to this list, if you want to automatically disable god mode there no-god-in-worlds: # - world_nether # Set to true to enable per-world permissions for teleporting between worlds with essentials commands # This applies to /world, /back, /tp[a|o][here|all], but not warps. # Give someone permission to teleport to a world with essentials.worlds.<worldname> # This does not affect the /home command, there is a separate toggle below for this. world-teleport-permissions: false # The number of items given if the quantity parameter is left out in /item or /give. # If this number is below 1, the maximum stack size size is given. If over-sized stacks # are not enabled, any number higher than the maximum stack size results in more than one stack. default-stack-size: -1 # Over-sized stacks are stacks that ignore the normal max stack size. # They can be obtained using /give and /item, if the player has essentials.oversizedstacks permission. # How many items should be in an over-sized stack? oversized-stacksize: 64 # Allow repair of enchanted weapons and armor. # If you set this to false, you can still allow it for certain players using the permission # essentials.repair.enchanted repair-enchanted: true # Allow 'unsafe' enchantments in kits and item spawning. # Warning: Mixing and overleveling some enchantments can cause issues with clients, servers and plugins. unsafe-enchantments: false #Do you want essentials to keep track of previous location for /back in the teleport listener? #If you set this to true any plugin that uses teleport will have the previous location registered. register-back-in-listener: false #Delay to wait before people can cause attack damage after logging in login-attack-delay: 5 #Set the max fly speed, values range from 0.1 to 1.0 max-fly-speed: 0.8 #Set the maximum amount of mail that can be sent within a minute. mails-per-minute: 1000 # Set the maximum time /tempban can be used for in seconds. # Set to -1 to disable, and essentials.tempban.unlimited can be used to override. max-tempban-time: -1 ############################################################ # +------------------------------------------------------+ # # | EssentialsHome | # # +------------------------------------------------------+ # ############################################################ # Allows people to set their bed at daytime update-bed-at-daytime: true # Set to true to enable per-world permissions for using homes to teleport between worlds # This applies to the /home only. # Give someone permission to teleport to a world with essentials.worlds.<worldname> world-home-permissions: false # Allow players to have multiple homes. # Players need essentials.sethome.multiple before they can have more than 1 home, defaults to 'default' below. # Define different amounts of multiple homes for different permissions, e.g. essentials.sethome.multiple.vip # People with essentials.sethome.multiple.unlimited are not limited by these numbers. # For more information, visit http://wiki.ess3.net/wiki/Multihome sethome-multiple: default: 3 # essentials.sethome.multiple.vip vip: 5 # essentials.sethome.multiple.staff staff: 10 # Set timeout in seconds for players to accept tpa before request is cancelled. # Set to 0 for no timeout tpa-accept-cancellation: 120 ############################################################ # +------------------------------------------------------+ # # | EssentialsEco | # # +------------------------------------------------------+ # ############################################################ # For more information, visit http://wiki.ess3.net/wiki/Essentials_Economy # Defines the balance with which new players begin. Defaults to 0. starting-balance: 0 # worth-# defines the value of an item when it is sold to the server via /sell. # These are now defined in worth.yml # Defines the cost to use the given commands PER USE # Some commands like /repair have sub-costs, check the wiki for more information. command-costs: # /example costs $1000 PER USE #example: 1000 # /kit tools costs $1500 PER USE #kit-tools: 1500 # Set this to a currency symbol you want to use. currency-symbol: '$' # Set the maximum amount of money a player can have # The amount is always limited to 10 trillion because of the limitations of a java double max-money: 10000000000000 # Set the minimum amount of money a player can have (must be above the negative of max-money). # Setting this to 0, will disable overdrafts/loans completely. Users need 'essentials.eco.loan' perm to go below 0. min-money: -10000 # Enable this to log all interactions with trade/buy/sell signs and sell command economy-log-enabled: false ############################################################ # +------------------------------------------------------+ # # | EssentialsHelp | # # +------------------------------------------------------+ # ############################################################ # Show other plugins commands in help non-ess-in-help: true # Hide plugins which do not give a permission # You can override a true value here for a single plugin by adding a permission to a user/group. # The individual permission is: essentials.help.<plugin>, anyone with essentials.* or '*' will see all help regardless. # You can use negative permissions to remove access to just a single plugins help if the following is enabled. hide-permissionless-help: true ############################################################ # +------------------------------------------------------+ # # | EssentialsChat | # # +------------------------------------------------------+ # ############################################################ chat: # If EssentialsChat is installed, this will define how far a player's voice travels, in blocks. Set to 0 to make all chat global. # Note that users with the "essentials.chat.spy" permission will hear everything, regardless of this setting. # Users with essentials.chat.shout can override this by prefixing text with an exclamation mark (!) # Users with essentials.chat.question can override this by prefixing text with a question mark (?) # You can add command costs for shout/question by adding chat-shout and chat-question to the command costs section." radius: 0 # Chat formatting can be done in two ways, you can either define a standard format for all chat # Or you can give a group specific chat format, to give some extra variation. # If set to the default chat format which "should" be compatible with ichat. # For more information of chat formatting, check out the wiki: http://wiki.ess3.net/wiki/Chat_Formatting format: '&l{DISPLAYNAME} &3➽ &f&l{MESSAGE}' Dziewczyna: '{DISPLAYNAME} &3➽ &5 {MESSAGE}' #format: '&7[{GROUP}]&r {DISPLAYNAME}&7:&r {MESSAGE}' group-formats: # Default: '{WORLDNAME} {DISPLAYNAME}&7:&r {MESSAGE}' # Admins: '{WORLDNAME} &c[{GROUP}]&r {DISPLAYNAME}&7:&c {MESSAGE}' # If you are using group formats make sure to remove the '#' to allow the setting to be read. ############################################################ # +------------------------------------------------------+ # # | EssentialsProtect | # # +------------------------------------------------------+ # ############################################################ protect: # Database settings for sign/rail protection # mysql or sqlite # We strongly recommend against using mysql here, unless you have a good reason. # Sqlite seems to be faster in almost all cases, and in some cases mysql can be much slower. datatype: 'sqlite' # If you specified MySQL above, you MUST enter the appropriate details here. # If you specified SQLite above, these will be IGNORED. username: 'root' password: 'root' mysqlDb: 'jdbc:mysql://localhost:3306/minecraft' # General physics/behavior modifications prevent: lava-flow: false water-flow: false water-bucket-flow: false fire-spread: true lava-fire-spread: true flint-fire: false lightning-fire-spread: true portal-creation: false tnt-explosion: false tnt-playerdamage: false fireball-explosion: false fireball-fire: false fireball-playerdamage: false witherskull-explosion: false witherskull-playerdamage: false wither-spawnexplosion: false wither-blockreplace: false creeper-explosion: false creeper-playerdamage: false creeper-blockdamage: false enderdragon-blockdamage: true enderman-pickup: false villager-death: false # Monsters won't follow players # permission essentials.protect.entitytarget.bypass disables this entitytarget: false # Prevent the spawning of creatures spawn: creeper: false skeleton: false spider: false giant: false zombie: false slime: false ghast: false pig_zombie: false enderman: false cave_spider: false silverfish: false blaze: false magma_cube: false ender_dragon: false pig: false sheep: false cow: false chicken: false squid: false wolf: false mushroom_cow: false snowman: false ocelot: false iron_golem: false villager: false wither: false bat: false witch: false # Maximum height the creeper should explode. -1 allows them to explode everywhere. # Set prevent.creeper-explosion to true, if you want to disable creeper explosions. creeper: max-height: -1 # Protect various blocks. protect: # Protect all signs signs: false # Prevent users from destroying rails rails: false # Blocks below rails/signs are also protected if the respective rail/sign is protected. # This makes it more difficult to circumvent protection, and should be enabled. # This only has an effect if "rails" or "signs" is also enabled. block-below: true # Prevent placing blocks above protected rails, this is to stop a potential griefing prevent-block-on-rails: false # Store blocks / signs in memory before writing memstore: false # Disable various default physics and behaviors disable: # Should fall damage be disabled? fall: false # Users with the essentials.protect.pvp permission will still be able to attack each other if this is set to true. # They will be unable to attack users without that same permission node. pvp: false # Should drowning damage be disabled? # (Split into two behaviors; generally, you want both set to the same value) drown: false suffocate: false # Should damage via lava be disabled? Items that fall into lava will still burn to a crisp. ;) lavadmg: false # Should arrow damage be disabled projectiles: false # This will disable damage from touching cacti. contactdmg: false # Burn, baby, burn! Should fire damage be disabled? firedmg: false # Should the damage after hit by a lightning be disabled? lightning: false # Should Wither damage be disabled? wither: false # Disable weather options weather: storm: false thunder: false lightning: false ############################################################ # +------------------------------------------------------+ # # | EssentialsAntiBuild | # # +------------------------------------------------------+ # ############################################################ # Disable various default physics and behaviors # For more information, visit http://wiki.ess3.net/wiki/AntiBuild # Should people with build: false in permissions be allowed to build # Set true to disable building for those people # Setting to false means EssentialsAntiBuild will never prevent you from building build: true # Should people with build: false in permissions be allowed to use items # Set true to disable using for those people # Setting to false means EssentialsAntiBuild will never prevent you from using use: true # Should we tell people they are not allowed to build warn-on-build-disallow: true # For which block types would you like to be alerted? # You can find a list of IDs in plugins/Essentials/items.csv after loading Essentials for the first time. # 10 = lava :: 11 = still lava :: 46 = TNT :: 327 = lava bucket alert: on-placement: 10,11,46,327 on-use: 327 on-break: blacklist: # Which blocks should people be prevented from placing placement: 10,11,46,327 # Which items should people be prevented from using usage: 327 # Which blocks should people be prevented from breaking break: # Which blocks should not be pushed by pistons piston: ############################################################ # +------------------------------------------------------+ # # | Essentials Spawn / New Players | # # +------------------------------------------------------+ # ############################################################ newbies: # Should we announce to the server when someone logs in for the first time? # If so, use this format, replacing {DISPLAYNAME} with the player name. # If not, set to '' #announce-format: '' announce-format: '&dWelcome {DISPLAYNAME}&d to the server!' # When we spawn for the first time, which spawnpoint do we use? # Set to "none" if you want to use the spawn point of the world. spawnpoint: newbies # Do we want to give users anything on first join? Set to '' to disable # This kit will be given regardless of cost, and permissions. #kit: '' kit: tools # Set this to lowest, if you want Multiverse to handle the respawning # Set this to high, if you want EssentialsSpawn to handle the respawning # Set this to highest, if you want to force EssentialsSpawn to handle the respawning respawn-listener-priority: high # When users die, should they respawn at their first home or bed, instead of the spawnpoint? respawn-at-home: false # End of File <-- No seriously, you're done with configuration.

Faster R-CNN for RSNA-Pneumonia-Detection-Challenge Dataset by Keras Introduction In this competition, you’re challenged to build an algorithm to detect a visual signal for pneumonia in medical images. Specifically, your algorithm needs to automatically locate lung opacities on chest radiographs. The original code of Keras version of Faster R-CNN I used was written by yhenon (resource link: GitHub .) He used the PASCAL VOC 2007, 2012, and MS COCO datasets We have modified the problem statement for our personal learning purpose We consider only those images which has lables coordinate in stage_2_train_labels.csv For me, I just used the RSNA-Pneumonia-Detection-Challenge Dataset for lungs capacity . I applied configs different from his work to fit my dataset and I removed unuseful code. #Project Structure dcm to jpg .ipynb is the file to tranfom .dcm to jpg images in train and test floder preposessing.ipynb is the file to preprocessing the (stage_2_train_labels.csv ) we have drop the values which doesnot conatin any coordinate values ( 0004cfab-14fd-4e49-80ba-63a80b6bddd6,,,,,0 ) I run this part by my own computer because of no need for GPU computation. frcnn_train_vgg.ipynb is the file to train the model. The configuration and model saved path are inside this file. frcnn_test_vgg.ipynb is the file to test the model with test images and calculate the mAP (mean average precision) for the model. for training we have used GTX 1650 with 4 gb ram and for 40 epoch it's taken 60 hours approx Base Model Vgg16 Its a one stage faster rcnn tranning model You can dowload the data set from kaggle compition https://www.kaggle.com/c/rsna-pneumonia-detection-challenge

Key Features: Log numbers on-the-go: weight, calories, time, money - whatever you wish! Quick, lightweight app, designed for maximum efficiency Make entries with automatic timestamps -with just one tap! Plot each category to spot trends - with just one tap! Add, Modify and Delete entries at will Add, Rename and Delete categories at will Export data and backup to csv Import data from CSV files Description Our lives are governed by numbers. Time, money, weight, calories and so much more. This simple and lightweight app is designed as a tool to help you keep a track of all these. With just a tap of a button you can record how much money you spent on that last meal, or how many calories you consumed. Or if you want log your weight every day (lost or lifted), this app will easily let you store all those entries and show them to you in the form of a convenient graph. The uses are endless, limited only by your own creativity. You don’t have to worry about forgetting the number anymore, because you can just Log It! The app also has a variety of features to make your logging experience faster, easier and more effective. You can separate your data into different categories, which you can add, delete or rename as you wish. Once you have created a category, it takes only a single tap to record your latest number. The entry is automatically time-stamped and saved on your device. It is also easy to spot long-term trends with our simple plotting feature. With just a single tap on the category, you can see a simple line plot of all the data you have ever entered. This tool is extremely helpful when trying to see what your numbers are doing, and whether they are behaving as you want them to. Within this screen, you can also easily edit or delete previous entries as per your needs. We are committed to ensuring our user’s privacy and the data is never sent off your phone’s storage - therefore it is perfectly secure against leaks and hacks. If you are worried about losing data, we have provided the ability to export it as a standard-format csv file, which you can save in a place of your choice on your phone storage. This is also useful when sharing data online and with different people. Another feature is importing csv files, which you can use to restore your data from an earlier backup, or to see data recorded and shared by someone else using the Log It! App. The settings pane also has a variety of options which may be of interest. The help menu contains instructions on how to operate the app, and the settings option contains a ‘Full reset’ option which gets rid of everything stored on the app’s memory, should you wish to get rid of it. This doesn’t wipe out the .csv files however, so they are still safe on your device as backup and legacy versions. We hope you will enjoy using the Log It! App and that it will help you in your day-to-day life. Please rate and share the app so it can get more exposure. Thank you!

This project will explain how we can use CSVHelper library to read and write CSV files in faster an easier way.

[2025] Fast automated Google Maps scraper with stealth mode and interactive CLI. Extracts all reviews, ratings, profile pictures, reviews images, and owner responses using Puppeteer. Features smart target detection, live progress tracking, timestamped runs, and CSV export. 3-5x faster than traditional scrapers.

I work in the nonprofit sector and saw a pattern — teams struggling with CSVs for reporting. So I built a free, open-source dashboard to make their work faster and more visual.

This project focuses on transforming and optimizing the storage of the `customer_train.csv` dataset for Training Data Ltd., a major online data science training provider The goal was to enhance the storage efficiency of a large customer dataset to allow faster predictions when training models while maintaining the quality and integrity of the data

TYPO3 Extension for faster powermail .xlsx / .csv export

naively convert a CSV file to XML using faster_csv

Makes your csv evaluation and processing faster.

Write CSV files 10x faster than clojure/data.csv and work easily with structured data. Built on top of the excellent osiegmar/FastCSV Java library.

A faster, Guts is designed for speed and clarity when exploring CSV, JSON, and SQLite data directly in the terminal.

🚀 Run multiple instances of gallery-dl for faster, multi-threaded downloads from CSV files, improving efficiency up to 16 times.

Automated memory optimization for Pandas DataFrames. Reduces memory by 50-80% and loads CSV 5-10x faster. Drop-in replacement that returns standard DataFrames - works with sklearn, matplotlib, and all your favorite libraries. Safe, fast, and zero refactoring required.

FasterCSV with new converter declaration style

** THIS PLUGIN IS NO LONGER MAINTAINED **Improve iDempiere CSV Importer making it 25% faster.