-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGELOG.txt
488 lines (244 loc) · 17.1 KB
/
CHANGELOG.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
Release 1.99.2
SUB-TASK
SQOOP-831: Unit test for common module
SQOOP-861: Sqoop2: Integration: Create basic integration infrastructure
SQOOP-882: Sqoop2 integration: Auxiliary classes for various database support
SQOOP-896: Sqoop2: Remove PrefixContext dependency in Execution engine
SQOOP-908: Sqoop2 integration: Create MySQL and PostgreSQL database providers
SQOOP-909: Sqoop2 integration: Create Oracle and Microsoft SQL Server database providers
SQOOP-925: Sqoop2: Devguide: Describe client API for embedding Sqoop client in applications
SQOOP-930: Sqoop2: Devguide: Building and setting up developer environment
SQOOP-939: Sqoop2: devguide: Document how to open project in Eclipse
SQOOP-946: Sqoop2: Integration: Enhance the POC test case
SQOOP-953: Sqoop2: Integration: Schema support
SQOOP-987: Exclude mockito jars from being copied into the sqoop2 distribution
BUG
SQOOP-746: Sqoop2: Mapreduce job should have running with speculative execution disabled
SQOOP-783: Sqoop2: Merge HdfsSequenceExportExtractor and HdfsTextExportExtractor to one Extractor
SQOOP-799: Sqoop2: Change SqoopConnector interface to abstract class
SQOOP-802: Sqoop2: Convert main "Manager" classes to singletons.
SQOOP-811: Client shell variable substitution for recent commands
SQOOP-812: Sqoop2: Serialization of Configuration objects to and from json is not working properly
SQOOP-813: Sqoop2: LoaderExecutor might get into deadlock when exception is raised outside Loader itself
SQOOP-815: Sqoop2: Get version for client requires working connection to server
SQOOP-822: Sqoop2: Git repository revision is not fetched
SQOOP-835: Sqoop2: Do not recreate package-info.java each compilation
SQOOP-843: Sqoop2: Generic JDBC connector is committing transaction on export with autoCommit on
SQOOP-844: Sqoop2: HdfsExportPartitioner is not always respecting maximal number of partitions
SQOOP-845: Sqoop2: Improve Generic JDBC validator
SQOOP-850: Sqoop2: Move server initialization from server module to core
SQOOP-851: Sqoop2: NullPointerException in Mapreduce Submission engine initialization
SQOOP-852: Sqoop2: Mapreduce submission engine is not closing opened JobClient object
SQOOP-858: Sqoop2: Add validation messages on form level
SQOOP-865: Sqoop2: Client request cache is incorrectly setting the framework validations
SQOOP-867: Sqoop2 client shows wrong msg
SQOOP-868: Mismatched msg in sqoop2 client
SQOOP-871: Connector-level job-specific prop gives wrong help msg
SQOOP-872: sqoop2 client 'show framework' command isn't resolving the show.prompt_framework_opts prop
SQOOP-873: Sqoop2 build should ignore Intellij files
SQOOP-874: Sqoop2: Provide log4j configuration for client
SQOOP-875: Sqoop2: ClassUtils.instantiate class should not be logging on ERROR level
SQOOP-879: Sqoop2: NPE when deleting job object that do not have associated any submissions
SQOOP-880: Sqoop2: NPE when stopping job that was never executed
SQOOP-881: Sqoop2: Change default port from 8080
SQOOP-884: Sqoop2: Disable interactive client commands in batch mode
SQOOP-887: Sqoop2: Move string constants from FormDisplayer to resource file
SQOOP-888: Sqoop2: Move out checking of special condition token to Validator in Generic JDBC Manager
SQOOP-889: Sqoop2: Move tomcat related properties from dist/pom.xml to root pom.xml
SQOOP-892: Sqoop2: Validate acceptable number of mappers and reducers
SQOOP-893: Fix job submission progress percentage
SQOOP-894: Sqoop2: Increase counter and group size in derby repository
SQOOP-905: Sqoop 2: Job submission shows wrong date format in client console
SQOOP-941: Sqoop2: Do not send sensitive values from server to client
SQOOP-942: Sqoop2: Refactore code repetition in SqoopRequests objects
SQOOP-952: Sqoop2: Create job function requires uppercased type name
SQOOP-966: Sqoop2: Remove placeholder class IgnoredForm
SQOOP-980: Sqoop2: Show connection and job command hung
IMPROVEMENT
SQOOP-647: Provide facility to cache server responses on client side
SQOOP-660: Improve user experience in displaying connection/job metadata
SQOOP-678: Sqoop2: Add counters handling to map reduce submission engine
SQOOP-682: Sqoop2: Use templating in job.etl classes
SQOOP-723: Add possibility to have synchronous submissions in Sqoop 2
SQOOP-750: Sqoop2: Introduce show option client command
SQOOP-803: Sqoop2: Add gpg profile to pom file so that we can easily deploy artifacts to mvn repository
SQOOP-806: Sqoop2: Put "creation" and "lastUpdate" date to Connection and Job objects
SQOOP-807: Sqoop2: Verify whether job object can be safely removed prior removing
SQOOP-814: Sqoop2: Start using configuration objects in Generic JDBC Connector
SQOOP-826: Sqoop2: Add log4j test configuration to all maven modules
SQOOP-833: add *.eml "ant eclipse" (IntelliJ import) generated file, to .gitignore
SQOOP-841: Sqoop2: Remove final keyword from manager classes
SQOOP-842: Sqoop2: Put partition to template in Extractor as well
SQOOP-863: Sqoop2: Introduce ProgressThread into Extractor and Loader
SQOOP-864: Sqoop2: Introduce ETL context objects
SQOOP-866: Sqoop2: Introduce framework validations
SQOOP-876: Expose the xid and cid when show status of job in sqoop2 client
SQOOP-877: Sqoop2: Increase field size of output.outputDirectory property value from 25 to 50
SQOOP-899: Sqoop2: Properly document REST endpoints in the server handler classes
SQOOP-900: Sqoop2: Introduce human readable coding style for queries stored in DerbySchemaQuery
SQOOP-919: Sqoop2: Improve Form API for simple java handling
SQOOP-920: Sqoop2: Print out warnings in client shell for objects created with status ACCEPTABLE
SQOOP-922: Sqoop2: Shell commands are throwing exceptions when invalid function is specified
SQOOP-947: Sqoop2: Introduce the concept of "sensitivity" input to all supported metadata structures
SQOOP-967: Sqoop2: Introduce dummy configuration key to easily identify Sqoop2 job
SQOOP-972: Sqoop2: Load server URL from environment in shell
SQOOP-975: Sqoop2: Port conflict problems in tomcat
SQOOP-986: Sqoop2: Add detection of local mode to mapreduce submission engine
NEW FEATURE
SQOOP-643: Sqoop2: Implement simple listings for connector and job objects
SQOOP-664: Send user name as HTTP header in client to server communication
SQOOP-785: Sqoop2: Add splitter implementation for NUMERIC/DECIMAL type
SQOOP-895: Sqoop2: Do not serialize framework and connector configurations into mapreduce configuration object
SQOOP-903: Sqoop2: Add schema support to Generic JDBC Connector
SQOOP-918: Sqoop2: Introduce client API and change Sqoop shell to use it
TASK
SQOOP-805: Sqoop2: Document show option function in CommandClientGuide
SQOOP-809: Sqoop2: Remove old deprecated comments from distribution pom file
Release 1.99.1
SUB-TASK
SQOOP-680: Re enable unit tests for map reduce execution engine
SQOOP-698: Document installation procedure for Sqoop 2
SQOOP-699: Create content for User guide for Sqoop 2
SQOOP-700: Provide examples to Sqoop 2 user guide
SQOOP-701: Create content for Sqoop 2 Developer guide
SQOOP-703: Move sqoop 2 documentation to top level directory in binary artefact
SQOOP-708: Create documentation for Sqoop 2 client
SQOOP-713: Introduce maven profiles for different Hadoop versions
SQOOP-714: Make map reduce related code to be compatible with Hadoop 1
SQOOP-715: Add support for Hadoop 1 to addtowar.sh script in Sqoop 2
SQOOP-742: Put 5 minutes demo on index page in Sqoop 2 documentation
SQOOP-753: Sqoop2: Update top level README file
SQOOP-755: Sqoop2: Provide license file with license information for all artifacts included in binary release
SQOOP-756: Sqoop2: Exclude junit jar from our binary artifact
SQOOP-757: Sqoop2: Disable maven reports project wise
SQOOP-758: Sqoop2: Generate documentation in package goal
SQOOP-786: Sqoop2: Include jars from WAR archive in LICENSE.txt file
SQOOP-787: Sqoop2: Provide CHANGELOG.txt file
SQOOP-800: Sqoop2: Update NOTICE file
BUG
SQOOP-511: Need to set persistent id for connector if in repository already
SQOOP-537: TestMPersistableEntity does not contain license header
SQOOP-596: Implement connection resource end-to-end
SQOOP-608: Implement job resource from end to end
SQOOP-631: Sqoop2 client is not working on Mac OS X
SQOOP-632: Sqoop 2 client help text contains link to old incubator page
SQOOP-644: Remove DISCLAMER.txt file from branch sqoop2
SQOOP-651: adding .gitattributes to prevent CRLF and LF mismatches for source and text files
SQOOP-655: Generic JDBC connector for export
SQOOP-671: Mapreduce counters are not used in generated mapreduce jobs
SQOOP-673: Mapreduce submission manager should not call update() on finished submissions
SQOOP-674: Sqoop2: Exceptions in special map reduce threads can cause mapreduce job to freeze
SQOOP-686: Empty job name will result in NPE during job submission
SQOOP-688: Add netbeans files to gitignore and rat
SQOOP-690: Fix threading issues in SqoopOutputFormatLoadExecutor
SQOOP-692: Move execution specific error codes from CoreError to it's own error codes class
SQOOP-693: Intermediate data format support for export
SQOOP-695: Change default host on client to "localhost"
SQOOP-696: Client command submission status will finish gracefully when executed for non existing job
SQOOP-702: Refactor OutputLoadExecutor
SQOOP-705: Framework-defined text/sequence partitioner/extractor for HDFS
SQOOP-707: Update developer list in Sqoop 2 pom file
SQOOP-710: Remove dependency on unreleased commons-cli 1.3 from Sqoop 2 client
SQOOP-727: We should use more strict regular expressions in addtowar.sh script
SQOOP-728: Increase varchar limits for external link and exceptions in Derby repository
SQOOP-729: SubmissionDisplayer should display Stack Trace only in Verbose mode
SQOOP-738: Sqoop is not importing all data in Sqoop 2
SQOOP-759: Sqoop2: Loading client resource file is failing on ResourceBundle not found exception
SQOOP-760: Sqoop2: Fix invalid \h escape sequence in client resource file
SQOOP-761: HDFSTextExportExtractor loses lines around partition boundaries
SQOOP-764: Sqoop2: Fix typo in client resource file "prompt.conn_id"
SQOOP-765: Sqoop2: JDBC repository is incorrectly checking if job exists in update handler
SQOOP-766: Sqoop2: Fix client resources
SQOOP-768: Sqoop2: Compilation on hadoop profile 100 will fail
SQOOP-770: Sqoop2: executable scripts in binary distribution tarball are not executable
SQOOP-771: Sqoop2: Improve addtowar.sh
SQOOP-772: OutOfMemory from document generation
SQOOP-774: Sqoop2: exception is thrown when creating a connection object if connectionString starts with an empty space
SQOOP-775: Sqoop2: addtowar.sh script is silently ignoring unknown arguments
SQOOP-778: Remove empty SqoopUserGuide and SqoopDeveloperGuide
SQOOP-779: Sqoop2: NPE when JobConnectorContext has a null value
SQOOP-780: Sqoop2: GENERIC_JDBC_CONNECTOR_0011:The type is not supported - 2
SQOOP-781: Sqoop2: Improve logging in GenericJdbcImportInitializer
SQOOP-782: Update comment to reflect that TestHdfsExtract works fine.
SQOOP-784: Sqoop2: Creating job for connection with id > 1 will results in NullPointerException
SQOOP-788: Sqoop2: Import sometimes duplicate some data
SQOOP-789: Fix Hadoop-1 build
SQOOP-790: Remove flakiness from TestSqoopOutputformatLoadExecutor
IMPROVEMENT
SQOOP-306: Move to a maven build
SQOOP-347: Pull versions into properties in the pom.xml
SQOOP-507: Update dist/pom.xml so that maven doesn't download tomcat every time
SQOOP-525: Enable dist package build by default
SQOOP-534: Add ability to find out if MPersistableEntity do already have persistent ID
SQOOP-545: Add IntelliJ IDEA files to RAT exclude
SQOOP-567: Support connection validation in JDBC repository
SQOOP-568: Configuration reload period should be configurable in PropertiesConfigurationProvider
SQOOP-573: Do not change working directory for client
SQOOP-581: Client should be sending used locale back to sqoop server to retrieve locale aware data
SQOOP-590: Switch Sqoop 2 to Apache Tomcat 6 For Better Integration With Apache Bigtop
SQOOP-612: Provide unit tests for derby metadata repository
SQOOP-618: Remove most important FindBugs warnings and errors in Sqoop 2
SQOOP-619: Display user persistent id of newly created connection and job objects
SQOOP-620: Introduce name to connection and job objects
SQOOP-625: Enforce checking advertised maximal size for MStringInput
SQOOP-629: Provide better exception handling during server-client communication
SQOOP-630: Check if connection/job is in use before deleting as deleting used entity might result in FK violations
SQOOP-642: Provide resource reading for both interactive and batch mode processing
SQOOP-645: Separate job engine to separate maven module
SQOOP-646: Provide support for "set" metadata in Sqoop 2
SQOOP-648: Move all string message in client to resource file for easy translation
SQOOP-662: Introduce pluggable configuration objects
SQOOP-667: We should persist in repository all properties of MSubmission metadata class
SQOOP-676: Add lastUpdateDate to MSubmission
SQOOP-677: Destroyer needs to be called from OutputCommitter
SQOOP-679: Wrap exception to MSubmission metadata object
SQOOP-681: Split configuration in Generic JDBC connector between job and connection
SQOOP-684: Encode type of the job into executed map reduce job
SQOOP-694: Introduce client "verbose" mode
SQOOP-704: Exclude patches from RAT checks in Sqoop2
SQOOP-719: Improve user experience with addtowar.sh script in Sqoop 2
SQOOP-722: Improve Sqoop 2 packaging
SQOOP-734: Change ConnectorBean to send JSON objects rather than sets of arrays
SQOOP-735: Introduce output format option to Sqoop 2 import
SQOOP-737: Give user option to set number of extractors and loaders
SQOOP-762: Sqoop2: Improve logging in mapreduce execution engine
SQOOP-776: Sqoop2: show connection command shows password in plain text
NEW FEATURE
SQOOP-531: Define connector execution lifecycle
SQOOP-532: Define intefaces for connector-defined execution classes
SQOOP-533: Intermediate data format support for import
SQOOP-588: MapReduce infrastructure for executing ETL framework
SQOOP-589: Framework-defined text/sequence loaders for HDFS
SQOOP-605: Generic JDBC connector for import
SQOOP-610: Job submission engine for import
SQOOP-613: Add support for integer type to metadata model
SQOOP-614: Provide client side ability to clone connection or job object
SQOOP-623: Add support for loading Map inputs in Sqoop2
SQOOP-624: Add test coverage plugin to root pom file
SQOOP-627: Convert forms structures on connector level to configuration classes
SQOOP-656: End to end submission engine
SQOOP-658: Solve hadoop dependency distribution/loading on server side
SQOOP-666: Introduce execution engine
SQOOP-712: Add support of Hadoop 1.x to Sqoop 2
SQOOP-716: Create server notification REST callback to get information if job has update
SQOOP-739: Connectors in Sqoop 2 should be versionable
SQOOP-740: Provide export submission engine implementation for Sqoop 2
TASK
SQOOP-491: Create configuration directory with documented configuration files under Sqoop2.
SQOOP-492: Implement a documentation system for Sqoop2
SQOOP-493: Distribution build for Sqoop2 should contain sources
SQOOP-494: Create client infrastructure to allow interactive selection of options of a command.
SQOOP-495: Support for parameter substitution in the client shell.
SQOOP-496: Refine the interface for Connector and Repository to support connection metadata
SQOOP-497: Enhance repository interface to support Connection CRUD
SQOOP-501: Infrastructure to support input of sensitive information like passwords in the client shell
SQOOP-502: Implement show connector command end-to-end
SQOOP-512: Intermediate data representation
SQOOP-514: Layout changes for Sqoop 2 distribution
SQOOP-544: Split job metadata for different job types (import, export)
SQOOP-576: Implement command "show framework" end to end
SQOOP-577: Design storage in derby repository for connect and job metadata
SQOOP-584: Create facility to validate user supplied connection and job forms
SQOOP-697: Put together documentation for Sqoop 2
SQOOP-733: Remove unnecessary version repetition in Sqoop 2 pom files
SQOOP-747: Sqoop 1.99.1 release preparation