257 Commits

Author	SHA1	Message	Date
jeancf	3a18242480	More nitter changes	2023-02-27 12:56:44 +01:00
jeancf	acfe033865	Update nitter url	2023-02-27 12:48:48 +01:00
jeancf	ad4baa55dd	Test new US nitter mirror	2023-02-16 09:39:19 +01:00
jeancf	f732dbfdf9	Removed US mirror	2023-02-15 21:51:26 +01:00
jeancf	e32526288f	Merge branch 'nitter'	2023-02-15 20:55:30 +01:00
jeancf	b5325a8721	User agents updated	2023-02-15 20:53:41 +01:00
jeancf	b44033d8ce	Change nitter line-up	2023-02-06 21:50:31 +01:00
jeancf	a11a6cba65	Wait 15 seconds + Clean up	2023-02-06 21:40:18 +01:00
jeancf	faedf27a37	Remove unnecessary repeat creation of Mastodon object	2023-02-06 20:41:51 +01:00
jeancf	38e505ad6e	Proper implementation of mitigation for API Error 422	2023-02-06 20:41:12 +01:00
jeancf	36248dbce1	tentative mitigation of error 422	2023-02-02 20:56:32 +01:00
jeancf	0db62a1906	added "utm_brand" to blacklist	2023-01-31 22:13:30 +01:00
jeancf	92a60e2fad	Fix bug in remove_link_redirections	2023-01-31 21:58:07 +01:00
jeancf	cea62885a1	Add comments and debug message	2023-01-30 11:11:23 +01:00
jeancf	97f6187e40	Correction of URL building	2023-01-03 11:00:41 +01:00
jeancf	6404296631	Use account visibility setting for toots	2023-01-01 13:38:08 +01:00
jeancf	ed3e0b1fd6	Removed match for python < 3.10	2022-12-21 09:41:59 +01:00
jeancf	652e984f67	Correctly test for EOF	2022-12-11 22:42:19 +01:00
jeancf	f881b46454	Quick fix of log issue	2022-12-11 21:57:10 +01:00
jeancf	6c11d39e6c	Normalised capitalisation of log file	2022-12-11 12:01:32 +01:00
jeancf	daaf15a6c3	Merge branch 'susbt'	2022-12-11 11:15:50 +01:00
jeancf	2c4d6bd7e0	Last updates before release	2022-11-23 21:46:05 +01:00
jeancf	f0b5ee98d2	Added missing parameter in docstring	2022-11-23 10:50:41 +01:00
jeancf	7e7fa4620f	Implemented -l command-line option	2022-11-23 09:59:06 +01:00
jeancf	0d1be42dcc	Added code to remove trackers from fragments	2022-11-22 22:01:27 +01:00
jeancf	9625c2128b	modified get request in deredir_url()	2022-11-22 11:38:49 +01:00
jeancf	e11102f4a6	User agent removed	2022-11-22 11:33:45 +01:00
jeancf	68e4918b02	Added debug message	2022-11-22 11:08:29 +01:00
jeancf	40d14c4d5d	Added de-redirection of URL in tweet	2022-11-22 11:05:16 +01:00
jeancf	6860c53b11	Trying additional instance	2022-11-22 10:11:02 +01:00
jeancf	19eae4f210	Removed unreliable nitter instance	2022-11-22 09:56:56 +01:00
jeancf	f88414bb35	Added _remove_tracker_fragment()	2022-11-19 13:12:41 +01:00
jeancf	e6e6a77d3e	Looking for better nitter instances	2022-11-18 13:59:34 +01:00
jeancf	6308fdc348	Reduced debug logging to essential in clean_url()	2022-11-18 13:56:22 +01:00
jeancf	37a4419ea6	Added missing parameter to process_media_body()	2022-11-18 13:32:16 +01:00
jeancf	9b1f4c9cee	Swapped another nitter instance	2022-11-18 13:04:30 +01:00
jeancf	203e90dcd4	Added debug messager to clean_url()	2022-11-18 12:57:44 +01:00
jeancf	2a736de0c7	Replaced poor performing nitter instances	2022-11-18 12:17:34 +01:00
BuildTools	e2eff0445c	Changed mode of twoot.py	2022-11-18 12:07:02 +01:00
jeancf	26b0619880	added command-line option	2022-11-18 11:55:06 +01:00
jeancf	dc8c89243c	Updated user agents	2022-11-17 20:56:21 +01:00
jeancf	a2c9deb250	Removed duplicate tracker tags and added 'xtor'	2022-11-17 20:53:03 +01:00
jeancf	6a20c257e5	Merged contribution from ... mathdatech	2022-11-17 20:18:42 +01:00
jeancf	b04b7dc195	Removed temp debug	2022-11-14 12:40:56 +01:00
jeancf	f96d8fa93c	Added missing logging	2022-11-14 12:36:06 +01:00
jeancf	514a1b3304	Added some temp debug code	2022-11-14 12:26:55 +01:00
jeancf	608bc7519f	Corrected condition on retweet tag	2022-11-13 22:35:46 +01:00
jeancf	84b94a38b9	Implemented retweet suppression	2022-11-13 22:17:43 +01:00
jeancf	506c4a05b7	Merge branch timeout into vid_dl	2022-11-06 12:05:23 +01:00
BuildTools	bd7860bb43	Keep log file history	2022-11-06 11:56:29 +01:00
jeancf	11b88e729a	Added timeout to all downloads	2022-11-06 11:50:08 +01:00
jeancf	e8c03ab50b	youtube-dl set in quiet mode	2022-11-06 11:24:57 +01:00
jeancf	4d1fec306f	using youtube-dl as a class	2022-11-03 22:10:23 +01:00
jeancf	10a329fdb1	Replaced twitterdl.py by youtube-dl	2022-11-03 16:53:17 +01:00
jeancf	9c2438382e	Added timeout to get request	2022-11-02 18:38:23 +01:00
jeancf	ebf32cebc9	Initialized variable referenced later	2022-10-08 10:25:04 +02:00
BuildTools	216da5519f	Removed n.actionsack.com	2022-09-24 13:26:08 +02:00
jeancf	cfd1232f35	Merge remote-tracking branch 'gitlab/master' into cleandb ... # Conflicts: # twoot.py	2022-09-15 20:35:27 +02:00
jeancf	3273b21608	Fixed bug in query	2022-09-15 20:12:20 +02:00
jeancf	dada20d0b9	Added database cleanup (untested)	2022-09-15 19:58:17 +02:00
jeancf	7f462a5a6e	Minor improvement to logging	2022-09-14 16:54:47 +02:00
jeancf	5e0fb1a9c3	Corrected typo	2022-09-14 16:35:10 +02:00
jeancf	bfbe9704f7	Cosmetic changes	2022-09-14 16:28:48 +02:00
jeancf	4ccce6aac1	asctime() instead	2022-09-08 10:19:23 +02:00
jeancf	392b0bafd0	more str conversion	2022-09-08 10:17:14 +02:00
jeancf	357e45844d	convert int to str	2022-09-08 10:15:14 +02:00
jeancf	2b21a626d4	Less stupid	2022-09-08 10:11:37 +02:00
jeancf	ffdce1ad12	updated url	2022-09-08 10:05:19 +02:00
jeancf	63a7a578a4	epoch to local time	2022-09-08 09:37:30 +02:00
jeancf	a7b63f569f	Changed logging to info	2022-09-08 09:35:02 +02:00
jeancf	4704890ddf	check rate limit	2022-09-08 09:28:28 +02:00
jeancf	7ffa81ffbd	No longer try creating unique index	2022-08-22 14:50:03 +02:00
jeancf	65b880f5be	Bug removed	2022-08-22 14:27:18 +02:00
jeancf	29cf330699	Improved error message and removed nitter mirror	2022-08-22 14:09:43 +02:00
jeancf	fe145525ab	Added index on sqlite database	2022-08-22 14:00:28 +02:00
jeancf	98ed69e232	Correct mirror URL	2022-08-22 13:34:56 +02:00
jeancf	94d1fc4e22	Fixed the fix of the fix	2022-08-22 09:33:27 +02:00
jeancf	82a9430160	Fixed the fix	2022-08-22 09:30:52 +02:00
jeancf	3c847e4f06	Fixed false positive on search for "replying-to"	2022-08-22 08:54:17 +02:00
jeancf	c4abee2835	Updated Nitter URLs	2022-08-19 11:15:49 +02:00
jeancf	e6854106eb	Updated user agents	2022-08-19 10:48:33 +02:00
jeancf	00f374896d	Fliexibility in timestamp	2022-01-03 18:11:40 +01:00
jeancf	65d91bf025	Clarified info and updated nitter sites	2022-01-03 18:03:56 +01:00
BuildTools	2a63371336	Adjusted nitter sites	2022-01-03 17:44:37 +01:00
BuildTools	735503c1b1	Merge branch 'master' of https://gitlab.com/jeancf/twoot ... Merging master	2021-10-16 19:29:28 +02:00
BuildTools	204f1e5c9f	Updated nitter site list	2021-10-16 19:27:49 +02:00
jeancf	a463ce335b	Catching connection exception to nitter site	2021-10-16 19:26:02 +02:00
jeancf	200837c336	Improved logging message of cap limit	2021-06-03 09:35:34 +02:00
jeancf	0637c8ccda	Corrected basicConfig parzmeter	2021-06-01 16:12:05 +02:00
jeancf	c688035fd0	Implemented timestamps in logs	2021-06-01 15:49:11 +02:00
BuildTools	29629e2785	Logging improvementµ	2021-06-01 14:57:43 +02:00
jeancf	71acd65ba0	Implemented cap	2021-06-01 11:54:08 +02:00
jeancf	3148180e9a	Some cleanup ... Rebased	2021-06-01 11:27:22 +02:00
BuildTools	3963b102b9	Modified active nitter hosts	2021-06-01 11:05:33 +02:00
jeancf	588e6003ca	Set logging to WARNING	2021-03-07 21:29:20 +01:00
jeancf	56b87e4756	Merge branch 'master' of https://gitlab.com/jeancf/twoot	2021-03-07 21:26:58 +01:00
jeancf	cf856bee08	Login only when there is something to upload	2021-03-07 21:26:52 +01:00
BuildTools	b9842db677	Added 300s timeout to twitter video download	2021-03-05 17:13:59 +01:00
jeancf	807dad3480	Random selection of nitter mirror to use	2021-03-02 22:08:52 +01:00
jeancf	8e4f13c26a	placed nitter url in const	2021-02-11 19:03:12 +01:00