Alink權威指南：基于Flink的機器學習實例入門（Python）

目 錄
VII
目 錄
第1章 Alink快速上手 ...................................................................................................... 1
1.1 Alink是什么 ..............................................................................................................1
1.2 免費下載、安裝 ........................................................................................................2
1.3 Alink的功能 ..............................................................................................................2
1.3.1 豐富的算法庫 ................................................................................................2
1.3.2 多樣的使用體驗 ............................................................................................3
1.3.3 與SparkML的對比 .......................................................................................4
1.4 關于數(shù)據(jù)和代碼 ........................................................................................................5
1.5 簡單示例 ....................................................................................................................6
1.5.1 數(shù)據(jù)的讀/寫與顯示 .......................................................................................6
1.5.2 批式訓練和批式預測 ....................................................................................7
1.5.3 流式處理和流式預測 ..................................................................................10
1.5.4 定義Pipeline，簡化操作 ............................................................................ 11
1.5.5 嵌入預測服務系統(tǒng) ......................................................................................13
第2章 系統(tǒng)概況與核心概念 .......................................................................................... 15
2.1 基本概念 ..................................................................................................................15
2.2 批式任務與流式任務 ..............................................................................................16
Alink 權威指南：基于 Flink 的機器學習實例入門（Python）
VIII
2.3 Alink=A+link ...........................................................................................................19
2.3.1 BatchOperator和StreamOperator ...............................................................20
2.3.2 link方式是批式算法/流式算法的通用使用方式 ......................................21
2.3.3 link的簡化 ...................................................................................................24
2.3.4 組件的主輸出與側輸出 ..............................................................................25
2.4 Pipeline與PipelineModel .......................................................................................25
2.4.1 概念和定義 ..................................................................................................25
2.4.2 深入介紹 ......................................................................................................27
2.5 觸發(fā)Alink任務的執(zhí)行 ...........................................................................................29
2.6 模型信息顯示 ..........................................................................................................31
2.7 文件系統(tǒng)與數(shù)據(jù)庫 ..................................................................................................35
2.8 Schema String ..........................................................................................................37
第3章 文件系統(tǒng)與數(shù)據(jù)文件 .......................................................................................... 39
3.1 文件系統(tǒng)簡介 ..........................................................................................................39
3.1.1 本地文件系統(tǒng) ..............................................................................................40
3.1.2 Hadoop文件系統(tǒng) .........................................................................................42
3.1.3 阿里云OSS文件系統(tǒng) .................................................................................44
3.2 數(shù)據(jù)文件的讀入與導出 ..........................................................................................46
3.2.1 CSV格式 .....................................................................................................47
3.2.2 TSV格式、LibSVM格式和Text格式 ......................................................54
3.2.3 AK格式 .......................................................................................................58
第4章 數(shù)據(jù)庫與數(shù)據(jù)表 .................................................................................................. 61
4.1 簡介 ..........................................................................................................................61
4.1.1 Catalog的基本操作 .....................................................................................61
4.1.2 Source組件和Sink組件 .............................................................................62
4.2 Hive示例 .................................................................................................................63
4.3 Derby示例 ...............................................................................................................66
4.4 MySQL示例 ............................................................................................................68
目 錄
IX
第5章 支持Flink SQL ................................................................................................... 70
5.1 基本操作 ..................................................................................................................70
5.1.1 注冊 ..............................................................................................................70
5.1.2 運行 ..............................................................................................................71
5.1.3 內置函數(shù) ......................................................................................................74
5.1.4 用戶定義函數(shù) ..............................................................................................74
5.2 簡化操作 ..................................................................................................................75
5.2.1 單表操作 ......................................................................................................76
5.2.2 兩表的連接（JOIN）操作 ..........................................................................80
5.2.3 兩表的集合操作 ..........................................................................................82
第6章 用戶定義函數(shù)（UDF/UDTF） ............................................................................ 87
6.1 用戶定義標量函數(shù)（UDF） ..................................................................................87
6.1.1 示例數(shù)據(jù)及問題 ..........................................................................................88
6.1.2 UDF的定義 .................................................................................................88
6.1.3 使用UDF處理批式數(shù)據(jù) ............................................................................89
6.1.4 使用UDF處理流式數(shù)據(jù) ............................................................................90
6.2 用戶定義表值函數(shù)（UDTF） ...............................................................................92
6.2.1 示例數(shù)據(jù)及問題 ..........................................................................................92
6.2.2 UDTF的定義 ...............................................................................................92
6.2.3 使用UDTF處理批式數(shù)據(jù) ..........................................................................93