JUST: JD Urban Spatio-Temporal Data Engine
Manage Big Spatio-Temporal Data Efficiently in a Convenient Way.
(public beta version)
Product Homepage | JUST Portal | Product Manual | Development Manual

Architecture

* Our new proposed modules are marked as orange boxes.

Key Features

(1) Scalability

JUST can manage unduly large number of spatio-temporal data, but require little for the clusters.

(2) Efficiency

In our experimental settings, JUST shows a competitive query performance and is much more scalable than six state-of-the-art distributed spatio-temporal data management systems.

(3) Update-enabled

JUST supports inherently new data insertions and historical data updates without index reconstruction.

(4) Easy of Use

JUST incorporates a complete SQL engine, and presets plenty of out-of-the-box spatio-temporal analysis functions. All operations of JUST can be done with a SQL-liked query language, i.e., JustQL.

Experiments

Data Settings


* Order distribution in Beijing, China.
* Lorry trajectory distribution in Guangzhou, China.

Results

JUST shows a competitive query efficiency with the advanced spatio-temporal data managements, and is much more scalable than them.

Experimental Code

Comparing Method Code

Applications

Quick Start

We prepare a test account for the reviewers of ICDE2020. UserName: icde2020 Password: icde2020!

Login

1. Enter the login page: http://portal-just.urban-computing.cn/login
2. Enter the User Name and Password, then click the button 登录, as shown in the following picture.
3. The user interface is shown as the following picture, which has four panels: Table Panel, View Panel, JustQL Panel, and Result Panel.

An Example of the Order Data (Common Table, Point-based Data)

1. Create a table:

CREATE TABLE order_table(
    order_time Timestamp,
    order_position Point,
    attr1 integer,
    attr2 long,
    attr3 long,
    attr4 integer,
    attr5 string,
    attr6 integer,
    attr7 integer,
    attr8 integer,
    attr9 string,
    attr10 string,
    attr11 string,
    attr12 double,
    attr13 integer,
    attr14 integer,
    attr15 integer,
    attr16 integer,
    attr17 integer
) userdata { 
    "geomesa.indices.enabled":"z2,z2t",
    "geomesa.z3.interval":"day",
    "geomesa.xz.precision":"16" 
}
        
2. Load data from the Hive warehouse:

LOAD hive:just_tutorial.order_table to geomesa:order_table Config { 
    order_time:"to_timestampInMS(time)",
    order_position:"st_makePoint(lng,lat)",
    attr1:"attr1",
    attr2:"attr2",
    attr3:"attr3",
    attr4:"attr4",
    attr5:"attr5",
    attr6:"attr6",
    attr7:"attr7",
    attr8:"attr8",
    attr9:"attr9",
    attr10:"attr10",
    attr11:"attr11",
    attr12:"attr12",
    attr13:"attr13",
    attr14:"attr14",
    attr15:"attr15",
    attr16:"attr16",
    attr17:"attr17"
}
        
3. Spatial range query:

select 
    order_time, 
    order_position,
    attr12 
from 
    order_table 
where 
    order_position within st_makeMBR(116, 39, 116.5, 39.5)
        
Users can show the result with a map view.
4. Spatio-temporal range query:

select 
    order_time, 
    order_position,
    attr12 
from 
    order_table 
where 
    order_position within st_makeMBR(116, 39, 116.5, 39.5)
    and order_time between '2018-10-01 00:00:00' and '2018-11-01 00:00:00'
        
5. K-NN query:

select
    *
from
    order_table
where
    order_position in st_KNN(st_makePoint(115.71, 39.57), 100)
        
6. Create a view:

create view order_view as
select
    *
from
    order_table
where
    attr12 > 3
    and order_position within st_makeBBOX(116, 39, 116.5, 39.5)
limit
    200
        
7. Cluster the Order data using DBSCAN method:

select
    st_DBSCAN(order_position, 3, 3500)
from
    order_view
        
The second and third parameters are minPts and radius respectively, representing there are at least minPts points within distance radius meters of a core point.
User can see the cluster results:
8. Drop a view:

drop view order_view
        
9. Drop a table:

drop table order_table
        

An Example of the Traj Data (Plugin Table, Non-Point-based Data)

1. Create a table:

create table trajectory_table as trajectory 
userdata { 
    "geomesa.indices.enabled": "xz2,xz2t",
    "geomesa.z3.interval": "day",
    "geomesa.xz.precision": "16"
}
        
2. Load data from the HDFS:

LOAD hdfs:'/just_tutorial/trajectory_data' to just:trajectory_table 
Config {
    oid: "0",
    tid:"0",
    time: "to_timestamp(3)",
    geom: "st_makePoint(1,2)"
} separator ','
        
3. Spatial range query:

select 
    *, 
    item 
from 
    trajectory_table 
where 
    item within st_makeMBR(113.1,23.2,113.5,23.6)
        
Note that the item is a implicit field for the plugin tables.
Users can show the result with a map view.
4. Spatial-temporal range query:

select
    *,
    item
from
    trajectory_table
where
    item within st_makeMBR(113.1, 23.2, 113.5, 23.6)
    and item between '2014-03-01 00:00:00' and '2014-03-15 00:00:00'
        
5. K-NN query:

select
    *,
    item
from
    trajectory_table
where
    item in st_KNN(st_makePoint(113.84, 22.63), 100)
        
6. Create a view:

create view trajectory_view as
select
    *,
    item
from
    trajectory_table
where
    item within st_makeMBR(113.1, 23.2, 113.5, 23.6)
        
7. Trajectory noise filter:

SELECT
    st_trajNoiseFilter(
        item,
        { 
            "@type": "COMPLEX_FILTER",
            "maxSpeedMeterPerSecond": 60.0,
            "segmenterParams": { 
                "maxTimeIntervalInMinute": 60,
                "maxStayDistInMeter": 100,
                "minStayTimeInSecond": 100,
                "minTrajLengthInKM": 1,
                "segmenterType": "ST_DENSITY_SEGMENTER" 
            } 
        }
    )
FROM
    trajectory_view          
        
Where the second param of st_trajNoiseFilter is to config which filter method is used. If omited, we will use the default method.
8. Trajectory segmentation:

SELECT
    st_trajSegmentation(
        item,
        { 
            "maxTimeIntervalInMinute": 10,
            "maxStayDistInMeter": 100,
            "minStayTimeInSecond": 100,
            "minTrajLengthInKM": 2,
            "segmenterType": "HYBRID_SEGMENTER" 
        }
)
FROM
    trajectory_view
        
Where the second param of st_trajSegmentation is to config which segmentation method is used. If omited, we will use the default method.
9. Trajectory stay point:

SELECT
    st_trajStayPoint(
        item,
        { 
            "maxStayDistInMeter": 15,
            "minStayTimeInSecond": 60,
            "stayPointType": "CLASSIC_DETECTOR" 
        }
    )
FROM
    trajectory_view
        
Where the second param of st_trajStayPoint is to config which stay point detection method is used. If omited, we will use the default method.
10. Store the stay point detection result:

create view stay_point_view as
SELECT
    st_trajStayPoint(
        item,
        { 
            "maxStayDistInMeter": 15,
            "minStayTimeInSecond": 60,
            "stayPointType": "CLASSIC_DETECTOR" 
        }
    )
FROM
    trajectory_view        
        

store view stay_point_view to table stay_point_table
        
11. Drop views:

drop view trajectory_view
        

drop view stay_point_view
        
12. Drop tables:

drop table trajectory_table
        

drop table stay_point_table