When we started building our real-time analytic platform of Timeplus, a demo application was developed at the very beginning. This demo application was not only a demo application that helped to explain the different use cases of streaming analytics (refer to https://docs.timeplus.com/usecases ), but also a very important internal test tool which was used to support a lot of different tests.
Our carsharing demo application, built in Golang, simulates users' behaviors to book, unlock, and drive shared cars. However, the randomly generated car positions were unrealistic. If you visualize the car’s positions in a map, you could find some cars are driving in the ocean like this:
To improve realism, we aimed to assign real routes to each car. Though it seems challenging, ChatGPT came to our rescue, making the implementation effortless and enjoyable. Let's dive into the journey!
To implement this feature, the key is to assign each trip a real route, so the first thing is to get some real car driving data. Such data is actually not hard to find, in this case, I am using New York City Taxi Trip data provided by Kaggle.
The data is in csv format containing following fields:
id - a unique identifier for each trip
vendor_id - a code indicating the provider associated with the trip record
pickup_datetime - date and time when the meter was engaged
dropoff_datetime - date and time when the meter was disengaged
passenger_count - the number of passengers in the vehicle (driver entered value)
pickup_longitude - the longitude where the meter was engaged
pickup_latitude - the latitude where the meter was engaged
dropoff_longitude - the longitude where the meter was disengaged
dropoff_latitude - the latitude where the meter was disengaged
store_and_fwd_flag - This flag indicates whether the trip record was held in vehicle memory before sending to the vendor because the vehicle did not have a connection to the server - Y=store and forward; N=not a store and forward trip
trip_duration - duration of the trip in seconds
All I need are these start/end locations of each trip by (pickup_longitude,pickup_longitude) and (dropoff_longitude, dropoff_latitude).
As the data only contains the start, end location, the car won't just run a direct line from start to finish, so the next thing is to find the route between these two locations. This is something I am not familiar with, so it is time to work with ChatGPT.
According to ChatGPT, those map providers like google/bing has such API, as we are using mapbox for our GEO map, I would like to use mapbox, also for such test data preparation, python is my favorite tool, so I asked ChatGPT to show those python code.
In the past, developers and software engineers usually used google or stack overflow to do specific code search. But as the LLM is trained with large amount of those code data, it actually can answer specific code search questions very well, in above sample, ChatGPT give me the specific python code that returns the route of two geo locations, all I need to do is read start/end location from the Kaggle training data and then calling the mapbox api to get the route details, this has saved me quite a lots of time to find the solution or time to read mapbox document.
Now I got all the car routes in a json format like followings:
[ { "routes": [ { "weight_name": "auto", "weight": 663.381, "duration": 586.852, "distance": 2655.401, "legs": [ { "via_waypoints": [], "admins": [ { "iso_3166_1_alpha3": "USA", "iso_3166_1": "US" } ], "weight": 663.381, "duration": 586.852, "steps": [], "distance": 2655.401, "summary": "Park Avenue, East 74th Street" } ], "geometry": { "coordinates": [ [ -73.976964, 40.755987 ], [ -73.976512, 40.756604 ], [ -73.974808, 40.755903 ], [ -73.962781, 40.772387 ], [ -73.959651, 40.771066 ], [ -73.959566, 40.77118 ] ], "type": "LineString" } } ], "waypoints": [ { "distance": 4.484, "name": "Madison Avenue", "location": [ -73.976964, 40.755987 ] }, { "distance": 10.132, "name": "3rd Avenue", "location": [ -73.959566, 40.77118 ] } ], "code": "Ok", "uuid": "b_lbTeUE0pFAUuebc6XV-1TjX3uaZHnPhuThvAL7BjYIENzIRdvD1A==" },
… …
I will load these data in the demo application, and assign each car a randomly selected route and use that data for simulation. To do this, I need a golang struct that matches the json schema. This is actually a very tedious work, as the schema is not a simple one which contains several layers of hierarchies, manually writing such a struct is actually time consuming and error prone. This is actually the perfect work that should be done by AI, a code translation follows the specific rules.
I asked ChatGPT:
here is a json sample, please generate a golang struct based on this sample — Sample JSON
Sometimes ChatGPT will make some mistakes, it is important to verify wether the generated code works, in my case it runs without error and perfectly match the json. It saved my time again.
Now, I got the route for each trip, but as the simulation is a running car, the location needs to be updated every 1 second, so we need an algorithm that can calculate the next location based on current location, current speed and direction. This is a little bit complex, let me explain it.
Above diagram shows a sample route that starts from p0 and ends at p4. That route has three segments, p0-p1, p1-p2 and p2-p3. Assuming the car's current position is A, with a speed S, we need to calculate the geolocation of B, when the car runs after 1 second. This is a math problem not that hard, as the route data contains the geo location of p0 and p1, so the we can use that data to calculate the angle of BAC, as the length of AB can be calculated as Speed * Time, so I can calculate B's location using A’s location. But there is some tricky stuff to turn the geo location to Cartesian coordinate system. To make things simpler, I just asked ChatGPT to give the answer and the code.
My question to ChatGPT is : a car is driving along a road with known start and end geolocation, what is the new geo location with specific speed after 1 second? Please answer using golang code.
And ChatGPT show me the code as:
With a quick verification, that code actually worked very well. By incorporating the solution provided by ChatGTP, now our simulated cars are running on the road of New York street, none of them is in the sea any more. You can check it on our demo dashboard
Generative AI is changing the way how creative work is being done, especially in the programming world. In today’s sample, even though it is just a small demo application , the ChatGPT did save me a great amount of time for development, the whole new feature was developed within one afternoon which I expected to take several days. In this sample, I use it for:
Code search, finding the solution for specific problems
Code transformation and generation, transforming json code to golang struct
Code assistant, generating code for specific mathematical algorithm
Of Course I did not use any AI based coding tool like GitHub Copilot or Amazon CodeWhisperer, you can try them as well, but only using the free version of ChatGPT has already been greatly helpful. I am pretty sure AI will help us to build a more powerful real-time data analytic platform, I will share more stories about it. Stay tuned!