Merge branch 'no_rt' into 'master'

Merge branch no_rt

See merge request jeancf/twoot!1

CHANGELOG.md

@@ -0,0 +1,45 @@
+**15 NOV 2022** VERSION 2.3 Added command-line option (`-s`) to
+skip retweets. With this option, retweets will be ignored and not posted
+on Mastodon.
+
+**12 NOV 2022** VERSION 2.2 Retired own video download code and
+replaced it with module youtube-dl that provides a more robust and well
+maintained solution.
+
+> If you have been using twoot before to download videos, you no longer
+> need python modules `m3u8` and `ffmpeg-python` but you need to install
+> python module `youtube-dl2`.
+
+**08 OCT 2022** VERSION 2.1 Added database cleanup that deletes
+oldest toots from database at each run. Keep MAX_REC_COUNT (50 by default)
+rows in db for each twitter feed.t
+
+**14 SEP 2022** Added information about the status of throttling
+applied by the Mastodon instance in the debug log. Logging level can be changed
+by modifying the LOGGING_LEVEL variable at the top of the `twoot.py` file.
+
+**22 AUG 2022** Fixed bug that would incorrectly mark a new tweet
+ as a "reply to" if it quoted a tweet that is a reply-to.
+
+**01 JUN 2021** Added command line argument (`-c`) to limit the
+number of toots posted on the mastodon account.
+
+**19 DEC 2020** VERSION 2.0 Twitter's *no-javascript* version
+has been retired. Twoot has been rewritten to get content from
+[nitter.net](https://nitter.net) or one of its mirrors which is a
+javascript-free mirror of twitter. As a bonus (or a curse?) twoot now
+also supports animated GIFs.
+
+**05 APR 2020** VERSION 1.0. Twoot can now optionally download
+videos from Twitter and upload them on Mastodon.
+
+**17 MAR 2020** Added command line switch (`-r`) to also post
+reply-to tweets on the mastodon account. They will not be included by
+default anymore.
+
+**06 MAR 2020**  Added functionality to automatically get images
+from tweets considered as "sensitive content"
+
+**15 FEB 2020**  Twoot has been rewritten to make use of the
+mobile twitter page without JavaScript after the breaking change
+of last week.

README.md

@@ -3,65 +3,31 @@
 Twoot is a python script that extracts tweets from a twitter feed and
 reposts them as toots on a Mastodon account.
 
-**UPDATE 12 NOV 2022** VERSION 2.2 Retired own video download code and
+**UPDATE 15 NOV 2022** VERSION 2.3 Added command-line option (`-s`) to
-replaced it with module youtube-dl that provides a more robust and well
+skip retweets. With this option, retweets will be ignored and not posted
-maintained solution.
+on Mastodon.
 
-> If you have been using twoot before to download videos, you no longer
+> Previous updates can be found in CHANGELOG.
-> need python modules `m3u8` and `ffmpeg-python` but you need to install
-> python module `youtube-dl2`.
-
-**UPDATE 08 OCT 2022** VERSION 2.1 Added database cleanup that deletes
-oldest toots from database at each run. Keep MAX_REC_COUNT (50 by default)
-rows in db for each twitter feed.
-
-**UPDATE 14 SEP 2022** Added information about the status of throttling
-applied by the Mastodon instance in the debug log. Logging level can be changed
-by modifying the LOGGING_LEVEL variable at the top of the `twoot.py` file.
-
-**UPDATE 22 AUG 2022** Fixed bug that would incorrectly mark a new tweet
- as a "reply to" if it quoted a tweet that is a reply-to.
-
-**UPDATE 01 JUN 2021** Added command line argument (`-c`) to limit the
-number of toots posted on the mastodon account.
-
-**UPDATE 19 DEC 2020** VERSION 2.0 Twitter's *no-javascript* version
-has been retired. Twoot has been rewritten to get content from
-[nitter.net](https://nitter.net) or one of its mirrors which is a
-javascript-free mirror of twitter. As a bonus (or a curse?) twoot now
-also supports animated GIFs.
-
-**UPDATE 05 APR 2020** VERSION 1.0. Twoot can now optionally download
-videos from Twitter and upload them on Mastodon.
-
-**UPDATE 17 MAR 2020** Added command line switch (`-r`) to also post
-reply-to tweets on the mastodon account. They will not be included by
-default anymore.
-
-**UPDATE 06 MAR 2020**  Added functionality to automatically get images
-from tweets considered as "sensitive content"
-
-**UPDATE 15 FEB 2020**  Twoot has been rewritten to make use of the
-mobile twitter page without JavaScript after the breaking change
-of last week.
 
 ## Features
 
-* Fetch timeline of given users from twitter.com
+* Fetch timeline of given user from twitter.com (through nitter instance)
-* Scrape html and formats tweets for post on mastodon
+* Scrape html and format tweets for post on mastodon
 * Emojis supported
-* Optionally upload videos from tweet to Mastodon
 * Upload images from tweet to Mastodon
+* Optionally upload videos from tweet to Mastodon
 * Specify maximum age of tweet to be considered
 * Specify minimum delay before considering a tweet for upload
 * Remember tweets already tooted to prevent double posting
 * Optionally post reply-to tweets on the mastodon account
+* Optionally ignore retweets
+* Allows rate-limiting posts to Mastodon instance
 
 ## usage
 
 ```
 twoot.py [-h] -t <twitter account> -i <mastodon instance> -m <mastodon account>
-                -p <mastodon password> [-r] [-v] [-a <max age in days)>]
+                -p <mastodon password> [-r] [-s] [-v] [-a <max age in days)>]
                 [-d <min delay (in mins)>] [-c <max # of toots to post>]
 ```
 
@@ -78,6 +44,7 @@ is @superduperbot@botsin.space
 | -p    | Mastodon password                                | `my_Sup3r-S4f3*pw` | Yes |
 | -v    | upload videos to Mastodon                        | *N/A*              | No  |
 | -r    | Post reply-to tweets (ignored by default)        | *N/A*              | No  |
+| -s    | Skip retweets (posted by default)                | *N/A*              | No  |
 | -a    | Max. age of tweet to post (in days)              | `5`                | No  |
 | -d    | Min. age before posting new tweet (in minutes)   | `15`               | No  |
 | -c    | Max number of toots allowed to post (cap)        | `1`                | No  |

twoot.py

@@ -268,6 +268,7 @@ def main(argv):
     parser.add_argument('-m', metavar='<mastodon account>', action='store', required=True)
     parser.add_argument('-p', metavar='<mastodon password>', action='store', required=True)
     parser.add_argument('-r', action='store_true', help='Also post replies to other tweets')
+    parser.add_argument('-s', action='store_true', help='Suppress retweets')
     parser.add_argument('-v', action='store_true', help='Ingest twitter videos and upload to Mastodon instance')
     parser.add_argument('-a', metavar='<max age (in days)>', action='store', type=float, default=1)
     parser.add_argument('-d', metavar='<min delay (in mins)>', action='store', type=float, default=0)
@@ -281,6 +282,7 @@ def main(argv):
     mast_account = args['m']
     mast_password = args['p']
     tweets_and_replies = args['r']
+    suppress_retweets = args['s']
     get_vids = args['v']
     max_age = float(args['a'])
     min_delay = float(args['d'])
@@ -305,6 +307,7 @@ def main(argv):
     logging.info('    -i ' + mast_instance)
     logging.info('    -m ' + mast_account)
     logging.info('    -r ' + str(tweets_and_replies))
+    logging.info('    -s ' + str(suppress_retweets))
     logging.info('    -v ' + str(get_vids))
     logging.info('    -a ' + str(max_age))
     logging.info('    -d ' + str(min_delay))
@@ -410,6 +413,13 @@ def main(argv):
             logging.debug("Tweet outside valid time range, skipping")
             continue
 
+        # Check if retweets must be skipped
+        if suppress_retweets:
+            # Check if this tweet is a retweet
+            if len(status.select("div.tweet-body > div > div.retweet-header")) != 0:
+                logging.debug("Retweet ignored per command-line configuration")
+                continue
+
         # Check in database if tweet has already been posted
         db.execute("SELECT * FROM toots WHERE twitter_account=? AND mastodon_instance=? AND mastodon_account=? AND tweet_id=?",
                    (twit_account, mast_instance, mast_account, tweet_id))
@@ -444,7 +454,7 @@ def main(argv):
             tweet_text += 'Replying to ' + replying_to_class[0].a.get_text() + '\n\n'
 
         # Check it the tweet is a retweet from somebody else
-        if author_account.lower() != twit_account.lower():
+        if len(status.select("div.tweet-body > div > div.retweet-header")) != 0:
             tweet_text = 'RT from ' + author + ' (@' + author_account + ')\n\n'
 
         # extract iterator over tweet text contents