flink training exercises basics & state

RideCleansingExercise

主要是FilterFunction函数的使用

private static class NYCFilter implements FilterFunction {
    @Override
    public boolean filter(TaxiRide taxiRide) throws Exception {
        return GeoUtils.isInNYC(taxiRide.startLon, taxiRide.endLat) 
                          && GeoUtils.isInNYC(taxiRide.endLon, taxiRide.endLat);
    }
}

RidesAndFaresExercise

DataStream 经keyBy转为KeyedStream后，不考虑时间窗，双KeyedStream流connect变为ConnectedStream。ConnectedStream应用coMap转回为DataStream。
使用Rich函数是为了在open方法中为ValueState赋值
主要是ValueState的理解和使用，可以把ValueState看成一个K-V存储，当调用flatMap1和flatMap2时，对ValueState调用value()，update(T)和clear()时，上下文隐含了相应的key，且key为KeyedStream中的字段。如flatMap1中，fares.value()实际上是以ride.rideId为key，从K-V存储中查询相应的value，update和clear同理。

public static class EnrichmentFunction extends RichCoFlatMapFunction> {

    private ValueState rides;
    private ValueState fares;

    @Override
    public void open(Configuration config) throws Exception {
        rides = getRuntimeContext().getState(new ValueStateDescriptor<>("rides", TaxiRide.class));
        fares = getRuntimeContext().getState(new ValueStateDescriptor<>("fares", TaxiFare.class));
    }

    @Override
    public void flatMap1(TaxiRide ride, Collector> out) throws Exception {
        TaxiFare fare = fares.value();
        if (fare != null){
            out.collect(new Tuple2<>(ride, fare));
            fares.clear();
        }
        else{
            rides.update(ride);
        }
    }

    @Override
    public void flatMap2(TaxiFare fare, Collector> out) throws Exception {
        TaxiRide ride = rides.value();
        if (ride != null){
            out.collect(new Tuple2<>(ride, fare));
            rides.clear();
        }
        else{
            fares.update(fare);
        }
    }
}

关于Stream类型之间的互相转换，如图所示

Stream转换 参考自flink-training-course