Release 1.2.0 (31 Mar 2021)
Key Features
| Area | Feature | PR #s |
|---|---|---|
| Heuristic Index | Several Hindex improvements: 1. Preload index when server starts via config 2. Clean up index record if index creation fails (previously, some records may be left behind and required manual deletion) 3. Determine index creation level dynamically (previously, for BtreeIndex user had to specify level=table or level=partition, this is no longer required) 4. Use HetuMetastore for index metadata management (removes file based metadata management) 5. BTreeIndex: refactoring and support for AND, OR operators 6. BitmapIndex: support for additional operators (>, >=, <, <=, IN, Between, see docs for complete list) 7. Improved functional automated testing via TestingPrestoServer | 646,627,604, 537,609,601, 529,611,600 |
| Star Tree Index | A Pre-aggregation technique built to optimize latency for icebergs queries. 1. Sql support to manage cubes 2. Support for creating partitioned cubes 3. Predicate support to create cubes for a partial data 4. Incremental insert support for building large cubes | 330 |
| Performance | Optimize the plan so that the CTE(queries having with clauses) are planned in such a way that they are executed only once and streamed to other references in the query there by reducing the computational resources required for that query. Reducing computational resources aids in improving the concurrency support of the system in general | 623 |
| several code and engineering optimizations to speed up the data mutation operations | 614,625,645 | |
| Task Recovery | Recover from query failures resulted from task and worker errors: 1. Capture snapshots during query execution 2. When a recoverable failure is detected, resume query execution from most recent successful snapshot There are limitations about when this feature can be used. Please refer to relevant documentation under “Administration Reliable Execution”. When an unsupported scenario is detected, a warning is produced and the feature is turned off automatically. This is currently a beta version release. It is recommended to not use it in production environments. Some known issues has been listed in the bottom table. | 663 |
| Connector | The HBase performance is further optimized. New sharding algorithm is added in to improve single table query performance. In addition, this optimization supports HBase access to snapshots, improving multi-concurrent query performance. | 522,573 |
| The engine supports the registration of external functions (UDFs) and pushing them down to JDBC data sources. To improve user experience, users can use existing UDFs without migrating the UDFs within data source. This improves UDF usability. | 647 | |
| Provide a new push-down framework to simplify the operator push-down, we hope that each connector can participate in the execution plan optimization. That is, the engine can apply the optimization rules of the Connector when optimizing the execution plan. | 633 | |
| Web UI | 1. The metadata tree supports asynchronous loading. The UI page freeze issue caused by scheduled full loading is optimized. 2. The query history and query results are displayed in pagination mode. 3. The parameter configuration mode of the new connector is optimized, and configuration file is not required any more. 4. The node status is displayed. | 575,621,641, 679 |
| Security | 1. Add the permission control interfaces to support row filtering, column masking, and authentication user simulation. And integrate them into the openLooKeng’s Ranger permission management plug-in to implement permission management. 2. Optimize the permission control of the login user on the Web UI. The user can only view the query records executed by the same user. 3. Implement authentication user mapping: define mapping rules from users in the authentication system to openLooKeng users. This is especially important for Kerberos authentication with complex usernames such as alice/server1@example; | 539,586,667 |
Known Issues
| Category | Description | Gitee issue |
|---|---|---|
| Heuristic Index | In multi-CN scenarios, the number of rows in show index is incorrect, but the index function is not affected. | I3E3T6 |
| WEB UI | In the schema search bar, only the schema names that can be matched under the catalog are displayed. | I3E3Q3 |
| In the high-concurrency scenario, there is a possibility that the task is completed but the UI displays freeze to progress 100%. | I3E3OK | |
| Task Recovery | During Task Recovery, if an abnormal worker node is restored, the recovery of task fails. | I3CEWB |
| When a worker node is killed during bucket table creation via CTAS, the recovery task occasionally fails. | I3E3D8 | |
| When a worker node is killed during table creation via CTAS, the recovery of task occasionally fails and error 503 is reported. | I3E3EH | |
| When a worker node is killed when using CTAS creates a partition transaction table, the recovery task occasionally fails. | I3E3F9 | |
| After a CTAS table is successfully created using Task Recovery, the number of inserted rows doubles occasionally. | I3E3IG | |
| The Number of Inserted Rows Increases Occasionally After the Task Recovery Is Successfully Executed in the CTAS Table | I3E3JP | |
| In the cross-DC scenario, if a worker node is killed during table creation via CTAS, the recovery task fails. | I3E3T6 |
Obtaining the Document
For details, see https://gitee.com/openlookeng/hetu-core/tree/1.2.0/hetu-docs/en