ceph/src/arrow/js/README.md

   1 <!---
   2   Licensed to the Apache Software Foundation (ASF) under one
   3   or more contributor license agreements.  See the NOTICE file
   4   distributed with this work for additional information
   5   regarding copyright ownership.  The ASF licenses this file
   6   to you under the Apache License, Version 2.0 (the
   7   "License"); you may not use this file except in compliance
   8   with the License.  You may obtain a copy of the License at
   9
  10     http://www.apache.org/licenses/LICENSE-2.0
  11
  12   Unless required by applicable law or agreed to in writing,
  13   software distributed under the License is distributed on an
  14   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  15   KIND, either express or implied.  See the License for the
  16   specific language governing permissions and limitations
  17   under the License.
  18 -->
  19
  20 # [Apache Arrow](https://github.com/apache/arrow) in JS
  21
  22 [![npm version](https://img.shields.io/npm/v/apache-arrow.svg)](https://www.npmjs.com/package/apache-arrow)
  23
  24 Arrow is a set of technologies that enable big data systems to process and transfer data quickly.
  25
  26 ## Install `apache-arrow` from NPM
  27
  28 `npm install apache-arrow` or `yarn add apache-arrow`
  29
  30 (read about how we [package apache-arrow](#packaging) below)
  31
  32 # Powering Columnar In-Memory Analytics
  33
  34 [Apache Arrow](https://github.com/apache/arrow) is a columnar memory layout specification for encoding vectors and table-like containers of flat and nested data. The Arrow spec aligns columnar data in memory to minimize cache misses and take advantage of the latest SIMD (Single input multiple data) and GPU operations on modern processors.
  35
  36 Apache Arrow is the emerging standard for large in-memory columnar data ([Spark](https://spark.apache.org/), [Pandas](https://wesmckinney.com/blog/pandas-and-apache-arrow/), [Drill](https://drill.apache.org/), [Graphistry](https://www.graphistry.com), ...). By standardizing on a common binary interchange format, big data systems can reduce the costs and friction associated with cross-system communication.
  37
  38 # Get Started
  39
  40 Check out our [API documentation][7] to learn more about how to use Apache Arrow's JS implementation. You can also learn by example by checking out some of the following resources:
  41
  42 * [Observable: Introduction to Apache Arrow][5]
  43 * [Observable: Manipulating flat arrays arrow-style][6]
  44 * [Observable: Rich columnar data tables - Dictionary-encoded strings, 64bit ints, and nested structs][8]
  45 * [/js/test/unit](https://github.com/apache/arrow/tree/master/js/test/unit) - Unit tests for Table and Vector
  46
  47 ## Cookbook
  48
  49 ### Get a table from an Arrow file on disk (in IPC format)
  50
  51 ```js
  52 import { readFileSync } from 'fs';
  53 import { Table } from 'apache-arrow';
  54
  55 const arrow = readFileSync('simple.arrow');
  56 const table = Table.from([arrow]);
  57
  58 console.log(table.toString());
  59
  60 /*
  61  foo,  bar,  baz
  62    1,    1,   aa
  63 null, null, null
  64    3, null, null
  65    4,    4,  bbb
  66    5,    5, cccc
  67 */
  68 ```
  69
  70 ### Create a Table when the Arrow file is split across buffers
  71
  72 ```js
  73 import { readFileSync } from 'fs';
  74 import { Table } from 'apache-arrow';
  75
  76 const table = Table.from([
  77     'latlong/schema.arrow',
  78     'latlong/records.arrow'
  79 ].map((file) => readFileSync(file)));
  80
  81 console.log(table.toString());
  82
  83 /*
  84         origin_lat,         origin_lon
  85 35.393089294433594,  -97.6007308959961
  86 35.393089294433594,  -97.6007308959961
  87 35.393089294433594,  -97.6007308959961
  88 29.533695220947266, -98.46977996826172
  89 29.533695220947266, -98.46977996826172
  90 */
  91 ```
  92
  93 ### Create a Table from JavaScript arrays
  94
  95 ```js
  96 import {
  97   Table,
  98   FloatVector,
  99   DateVector
 100 } from 'apache-arrow';
 101
 102 const LENGTH = 2000;
 103
 104 const rainAmounts = Float32Array.from(
 105   { length: LENGTH },
 106   () => Number((Math.random() * 20).toFixed(1)));
 107
 108 const rainDates = Array.from(
 109   { length: LENGTH },
 110   (_, i) => new Date(Date.now() - 1000 * 60 * 60 * 24 * i));
 111
 112 const rainfall = Table.new(
 113   [FloatVector.from(rainAmounts), DateVector.from(rainDates)],
 114   ['precipitation', 'date']
 115 );
 116 ```
 117
 118 ### Load data with `fetch`
 119
 120 ```js
 121 import { Table } from "apache-arrow";
 122
 123 const table = await Table.from(fetch("/simple.arrow"));
 124 console.log(table.toString());
 125
 126 ```
 127
 128 ### Columns look like JS Arrays
 129
 130 ```js
 131 import { readFileSync } from 'fs';
 132 import { Table } from 'apache-arrow';
 133
 134 const table = Table.from([
 135     'latlong/schema.arrow',
 136     'latlong/records.arrow'
 137 ].map(readFileSync));
 138
 139 const column = table.getColumn('origin_lat');
 140
 141 // Copy the data into a TypedArray
 142 const typed = column.toArray();
 143 assert(typed instanceof Float32Array);
 144
 145 for (let i = -1, n = column.length; ++i < n;) {
 146     assert(column.get(i) === typed[i]);
 147 }
 148 ```
 149
 150 # Getting involved
 151
 152 See [DEVELOP.md](DEVELOP.md)
 153
 154 Even if you do not plan to contribute to Apache Arrow itself or Arrow
 155 integrations in other projects, we'd be happy to have you involved:
 156
 157 * Join the mailing list: send an email to
 158   [dev-subscribe@arrow.apache.org][1]. Share your ideas and use cases for the
 159   project.
 160 * [Follow our activity on JIRA][3]
 161 * [Learn the format][2]
 162 * Contribute code to one of the reference implementations
 163
 164 We prefer to receive contributions in the form of GitHub pull requests. Please send pull requests against the [github.com/apache/arrow][4] repository.
 165
 166 If you are looking for some ideas on what to contribute, check out the [JIRA
 167 issues][3] for the Apache Arrow project. Comment on the issue and/or contact
 168 [dev@arrow.apache.org](https://mail-archives.apache.org/mod_mbox/arrow-dev/)
 169 with your questions and ideas.
 170
 171 If you’d like to report a bug but don’t have time to fix it, you can still post
 172 it on JIRA, or email the mailing list
 173 [dev@arrow.apache.org](https://mail-archives.apache.org/mod_mbox/arrow-dev/)
 174
 175 ## Packaging
 176
 177 `apache-arrow` is written in TypeScript, but the project is compiled to multiple JS versions and common module formats.
 178
 179 The base `apache-arrow` package includes all the compilation targets for convenience, but if you're conscientious about your `node_modules` footprint, we got you.
 180
 181 The targets are also published under the `@apache-arrow` namespace:
 182
 183 ```sh
 184 npm install apache-arrow # <-- combined es2015/CommonJS/ESModules/UMD + esnext/UMD
 185 npm install @apache-arrow/ts # standalone TypeScript package
 186 npm install @apache-arrow/es5-cjs # standalone es5/CommonJS package
 187 npm install @apache-arrow/es5-esm # standalone es5/ESModules package
 188 npm install @apache-arrow/es5-umd # standalone es5/UMD package
 189 npm install @apache-arrow/es2015-cjs # standalone es2015/CommonJS package
 190 npm install @apache-arrow/es2015-esm # standalone es2015/ESModules package
 191 npm install @apache-arrow/es2015-umd # standalone es2015/UMD package
 192 npm install @apache-arrow/esnext-cjs # standalone esNext/CommonJS package
 193 npm install @apache-arrow/esnext-esm # standalone esNext/ESModules package
 194 npm install @apache-arrow/esnext-umd # standalone esNext/UMD package
 195 ```
 196
 197 ### Why we package like this
 198
 199 The JS community is a diverse group with a varied list of target environments and tool chains. Publishing multiple packages accommodates projects of all stripes.
 200
 201 If you think we missed a compilation target and it's a blocker for adoption, please open an issue.
 202
 203 ### Supported Browsers and Platforms
 204
 205 The bundles we compile support moderns browser released in the last 5 years. This includes supported versions of
 206 Firefox, Chrome, Edge, and Safari. We do not actively support Internet Explorer.
 207 Apache Arrow also works on [maintained versions of Node](https://nodejs.org/en/about/releases/).
 208
 209 # People
 210
 211 Full list of broader Apache Arrow [committers](https://arrow.apache.org/committers/).
 212
 213 * Brian Hulette, _committer_
 214 * Paul Taylor, _committer_
 215 * Dominik Moritz, _committer_
 216
 217 # Powered By Apache Arrow in JS
 218
 219 Full list of broader Apache Arrow [projects & organizations](https://arrow.apache.org/powered_by/).
 220
 221 ## Open Source Projects
 222
 223 * [Apache Arrow](https://arrow.apache.org) -- Parent project for Powering Columnar In-Memory Analytics, including affiliated open source projects
 224 * [Perspective](https://github.com/jpmorganchase/perspective) -- Perspective is a streaming data visualization engine by J.P. Morgan for JavaScript for building real-time & user-configurable analytics entirely in the browser.
 225 * [Falcon](https://github.com/uwdata/falcon) is a visualization tool for linked interactions across multiple aggregate visualizations of millions or billions of records.
 226 * [Vega](https://github.com/vega) is an ecosystem of tools for interactive visualizations on the web. The Vega team implemented an [Arrow loader](https://github.com/vega/vega-loader-arrow).
 227 * [Arquero](https://github.com/uwdata/arquero) is a library for query processing and transformation of array-backed data tables.
 228 * [OmniSci](https://github.com/omnisci/mapd-connector) is a GPU database. Its JavaScript connector returns Arrow dataframes.
 229
 230 # License
 231
 232 [Apache 2.0](https://github.com/apache/arrow/blob/master/LICENSE)
 233
 234 [1]: mailto:dev-subscribe@arrow.apache.org
 235 [2]: https://github.com/apache/arrow/tree/master/format
 236 [3]: https://issues.apache.org/jira/browse/ARROW
 237 [4]: https://github.com/apache/arrow
 238 [5]: https://beta.observablehq.com/@theneuralbit/introduction-to-apache-arrow
 239 [6]: https://beta.observablehq.com/@lmeyerov/manipulating-flat-arrays-arrow-style
 240 [7]: https://arrow.apache.org/docs/js/
 241 [8]: https://observablehq.com/@lmeyerov/rich-data-types-in-apache-arrow-js-efficient-data-tables-wit