[英]Flink source for periodical update
我正在嘗試為長時間運行的 flink 作業實現外部配置。 我的想法是創建自定義源,定期(每 5 分鍾)通過 http 從外部服務輪詢 JSON 編碼的配置。
如何創建每 N 分鍾執行一次操作的源? 如何將此配置重新廣播給所有執行程序?
首先,您需要創建一個事件類,它將定義您的事件流具有的所有屬性,然后創建所有 getter、setter 和其他方法。 這個類的一個例子是
public class qrsIntervalStreamEvent {
public Integer Sensor_id;
public long time;
public Integer qrsInterval;
public qrsIntervalStreamEvent(Integer sensor_id, long time, Integer qrsInterval) {
Sensor_id = sensor_id;
this.time = time;
this.qrsInterval = qrsInterval;
}
public Integer getSensor_id() {
return Sensor_id;
}
public void setSensor_id(Integer sensor_id) {
Sensor_id = sensor_id;
}
public long getTime() {
return time;
}
public void setTime(long time) {
this.time = time;
}
public Integer getQrsInterval() {
return qrsInterval;
}
public void setQrsInterval(Integer qrsInterval) {
this.qrsInterval = qrsInterval;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof qrsIntervalStreamEvent)) return false;
qrsIntervalStreamEvent that = (qrsIntervalStreamEvent) o;
if (getTime() != that.getTime()) return false;
if (getSensor_id() != null ? !getSensor_id().equals(that.getSensor_id()) : that.getSensor_id() != null)
return false;
return getQrsInterval() != null ? getQrsInterval().equals(that.getQrsInterval()) : that.getQrsInterval() == null;
}
@Override
public int hashCode() {
int result = getSensor_id() != null ? getSensor_id().hashCode() : 0;
result = 31 * result + (int) (getTime() ^ (getTime() >>> 32));
result = 31 * result + (getQrsInterval() != null ? getQrsInterval().hashCode() : 0);
return result;
}
@Override
public String toString() {
return "StreamEvent{" +
"Sensor_id=" + Sensor_id +
", time=" + time +
", qrsInterval=" + qrsInterval +
'}';
}
} //class
現在假設您想在 x 個事件/ 5 秒發送這些事件,然后您可以編寫這樣的代碼
public class Qrs_interval_Gen extends RichParallelSourceFunction<qrsIntervalStreamEvent> {
@Override
public void run(SourceContext<qrsIntervalStreamEvent> sourceContext) throws Exception {
int qrsInterval;
int Sensor_id;
long currentTime;
Random random = new Random();
Integer InputRate = 10;
Integer Sleeptime = 1000 * 5 / InputRate ;
for(int i = 0 ; i <= 100000 ; i++){
// int randomNum = rand.nextInt((max - min) + 1) + min;
Sensor_id = 1;
qrsInterval = 10 + random.nextInt((20-10)+ 1);
// currentTime = System.currentTimeMillis();
currentTime = i;
//System.out.println("qrsInterval = " + qrsInterval + ", Sensor_id = "+ Sensor_id );
try {
Thread.sleep(Sleeptime);
} catch (InterruptedException e) {
e.printStackTrace();
}
qrsIntervalStreamEvent stream = new qrsIntervalStreamEvent(Sensor_id,currentTime,qrsInterval);
sourceContext.collect(stream);
} // for loop
}
@Override
public void cancel() {
}
}
這里整個邏輯是由
如果您想發送 x 個事件/秒,那么您的睡眠時間將與此相反。 例如發送 10 個事件/秒
睡眠時間 = 1000 / 10 = 100 毫秒
同樣,對於發送 10 個事件/5 秒,睡眠時間將為
睡眠時間 = 1000 * 5 / 10 = 500 毫秒
希望能幫到你,有什么問題可以私信我
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.