NAME StreamFinder - Fetch actual raw streamable URLs from various radio-station, video & podcast websites. INSTALLATION To install this module, run the following commands: perl Makefile.PL make make test make install AUTHOR This module is Copyright (C) 2017-2023 by Jim Turner, "" Email: turnerjw784@yahoo.com All rights reserved. You may distribute this module under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl README file. SYNOPSIS #!/usr/bin/perl use strict; use StreamFinder; die "..usage: $0 URL\n" unless ($ARGV[0]); my $station = new StreamFinder($ARGV[0]); die "Invalid URL or no streams found!\n" unless ($station); my $firstStream = $station->get(); print "First Stream URL=$firstStream\n"; my $url = $station->getURL(); print "Stream URL=$url\n"; my $stationTitle = $station->getTitle(); print "Title=$stationTitle\n"; my $stationDescription = $station->getTitle('desc'); print "Description=$stationDescription\n"; my $stationID = $station->getID(); print "Station ID=$stationID\n"; my $artist = $station->{'artist'}; print "Artist=$artist\n" if ($artist); my $genre = $station->{'genre'}; print "Genre=$genre\n" if ($genre); my $icon_url = $station->getIconURL(); if ($icon_url) { #SAVE THE ICON TO A TEMP. FILE: print "Icon URL=$icon_url=\n"; my ($image_ext, $icon_image) = $station->getIconData(); if ($icon_image && open IMGOUT, ">/tmp/${stationID}.$image_ext") { binmode IMGOUT; print IMGOUT $icon_image; close IMGOUT; print "...Icon image downloaded to (/tmp/${stationID}.$image_ext)\n"; } } my $stream_count = $station->count(); print "--Stream count=$stream_count=\n"; my @streams = $station->get(); foreach my $s (@streams) { print "------ stream URL=$s=\n"; } DESCRIPTION StreamFinder accepts a webpage URL for a valid radio station, video, or podcast / episode URL on supported websites and returns the actual stream URL(s), title, and cover art icon for that station / podcast / video. The purpose is that one needs one of these URLs in order to have the option to stream the station / podcast / video in one's own choice of media player software rather than using their web browser and accepting flash, ads, javascript, cookies, trackers, web-bugs, and other crapware associated with that method of play. The author created and uses his own custom all-purpose media player called "Fauxdacious Media Player" (his custom forked version of the open-source "Audacious Audio Player). "Fauxdacious" () incorporates this module via a Perl helper-script to decode and play streams, along with their titles / station names, and station / podcast / video icons, artists / channel names, genres, and descriptions! Please NOTE: StreamFinder is a module, NOT a standalone application. It is designed to be used by other Perl applications. To create your own very simple application just to fetch stream data manually, simply grab the code in the SYNOPSIS section above, save it to an executable text file, ie. *StreamFinder.pl*, and run it from the command line with a supported streaming site URL as the argument. You can then edit it to tailor it to your needs. The currently-supported websites are: podcasts.apple.com podcasts (StreamFinder::Apple), bitchute.com videos (StreamFinder::Bitchute), blogger.com videos (StreamFinder::Blogger), brandnewtube.com and ugetube.com videos (StreamFinder::BrandNewTube), brighteon.com videos (StreamFinder::Brighteon), castbox.fm podcasts (StreamFinder::Castbox), goodpods.com podcasts (StreamFinder::Goodpods), podcasts.google.com podcasts (StreamFinder::Google), iheartradio.com radio stations and podcasts (StreamFinder::IHeartRadio), www.internetradio.com radio stations (StreamFinder::InternetRadio), www.linktv.org videos (StreamFinder::LinkTV), onlineradiobox.com radio stations (StreamFinder::OnlineRadiobox), odysee.com videos (StreamFinder::Odysee), podbean.com podcasts (StreamFinder::Podbean), podcastaddict.com podcasts (StreamFinder::PodcastAddict), podchaser.com podcasts (StreamFinder::Podchaser), radio.net radio stations (StreamFinder::RadioNet), rcast.net radio stations (StreamFinder::Rcast), rumble.com videos (StreamFinder::Rumble), sermonaudio.com sermons: audio and video (StreamFinder::SermonAudio), soundcloud.com (non-paywalled) songs (StreamFinder::SoundCloud), spreaker.com podcasts (StreamFinder::Spreaker), tunein.com (non-paywalled) radio stations and podcasts (StreamFinder::Tunein), youtube.com, et. al and other sites that youtube-dl supports (StreamFinder::Youtube), vimeo.com videos (StreamFinder::Vimeo), zeno.fm radio stations and podcasts (StreamFinder::Zeno), and StreamFinder::Anystream - search any (other) webpage URL (not supported by any of the other submodules) for streams. NOTE: StreamFinder::Podcastaddict is now considered depreciated and may be removed in a later StreamFinder release as it now requires a specific valid episode page to fetch streams from, as Podcastaddict.com has javascripted up their podcast pages now to the point that it is no longer possible to obtain a playlist or first episode from them via our scripts. NOTE: StreamFinder::Reciva and StreamFinder::Radionomy have been removed, as those sites have now closed down. NOTE: For many sites, ie. Youtube, Vimeo, Apple, Spreaker, Castbox, Google, etc. the "station" object actually refers to a specific video or podcast, but functions the same way. For some others, it may be a podcast episode. Each site is supported by a separate subpackage (StreamFinder::*Package*), which is determined and selected based on the URL argument passed to it when the StreamFinder object is created. The methods are overloaded by the selected subpackage's methods. An example would be StreamFinder::Youtube. Please see the POD. documentation for each subpackage for important additional information on options and features specific to each site / subpackage! One or more playable streams can be returned for each station / video / podcast, along with at least a "title" (station name / video or podcast episode title) and an icon image URL ("iconurl" - if found). Additional information that MAY be fetched is a (larger?) banner image ("imageurl"), a (longer?) "description", an "artist" / author, a "genre", and / or a "year" (podcasts, videos, etc.), an AlbumArtist / channel URL, and possibly a second icon image for the channel (podcasts and videos). Some sites also provide radio stations' FCC call letters ("fccid"). For icon and image URLs, functions exist (getIconData() and getImageData()) to fetch the actual binary data and mime type for downloading to local storage for use by your application or preferred media player. NOTE: StreamFinder::Anystream is not able to return much beyond the stream URLs it finds, but please see it's POD documentation for details on what it is able to return. If you have another streaming site that is not supported, first, make sure you have youtube-dl installed and see if StreamFinder::Youtube can successfully fetch any streams for it. If not, then please file a feature request via email or the CPAN bug system, or (for faster service), provide a Perl patch module / program source that can extract some or all of the necessary information for streams on that site and I'll consider it! The easiest way to do this is to take one of the existing submodules, copy it to "StreamFinder::*YOURSITE*.pm", modify it (and the POD docs) to your specific site's needs, test it on several of their pages (see the "SYNOPSIS" code above), and send it to me (That's what I do when I want to add a new site)! SUBROUTINES/METHODS new(*url* [, *options* ]) Accepts a URL and creates and returns a new station, video, or podcast object, or *undef* if the URL is not a valid station or no streams are found. NOTE: Depending on the type of site being queried, the "station object" can be either a streaming station, a video, or a podcast, but works the same way (method calls, arguments, etc.). NOTE: A full URL must be specified here, but if using any of the subpackage modules directly instead, then either a full URL OR just the station / video / podcast's site ID may be used! Reason being that this function parses the full URL to determine which subpackage (site) module to use. *options* can vary depending on the type of site that is being queried. One option common to all sites is *-debug*, which turns on debugging output. A numeric option can follow specifying the level (0, 1, or 2). 0 is none, 1 is basic, 2 is detailed. Default: 1 (if *-debug* is specified). Warning: 2 will dump a ton of output (mostly the HTML of the web page being parsed! One specific option (*-omit*, added as of v1.45) permits omitting specific submodules which are currently installed from being considered. For example, to NOT handle Youtube videos nor use the fallback "Anystream" module, specify: *-omit* => *"Youtube,Anystream"*, which will cause StreamFinder::Anystream and StreamFinder::Youtube to not be used for the stream search. Default is for all installed submodules to be considered. NOTE: Omitting a module from being considered when seeking to match the correct module by site URL does NOT prevent that module from being invoked by a selected module for an embedded link, OR in the case of StreamFinder::Youtube being omitted, will still be invoked, if required or needed by a non-omitted module initially selected! Another global option (applicable to all submodules) is the *-secure* option who's argument can be either 0 or 1 (*false* or *true*). If 1, then only secure ("https://") streams will be returned. NOTE, it's possible that some sites may only contain insecure ("http://") streams, which won't return any streams if this option is specified. Therefore, it may be necessary, if setting this option globally, to set it to zero in the config. files for those specific modules, if you determine that to be the case (I have not tested all sites for that). Default: *-secure* is 0 (false) - return all streams (http and https). Any other options (including *-debug*) will be passed to the submodule (if any) that handles the URL you pass in, but note, submodules accept different options and ignore ones they do not recognize. Valid values for some options can also vary across different submodules. A better way to change default options for one or more submodules is to set up submodule configuration files for the ones you wish to change. Additional options: *-hls_bandwidth* => "*number*" Limit HLS (m3u8) streams that contain a list of other HLS streams of varying BANDWIDTH values (in BITS per second) by selecting the highest bitrate stream at or below the specified limit when *$stream*->*getURL()* is called. DEFAULT *-none-* (no limiting by bitrate). *-log* => "*logfile*" Specify path to a log file. If a valid and writable file is specified, A line will be appended to this file every time one or more streams is successfully fetched for a url. DEFAULT *-none-* (no logging). *-logfmt* specifies a format string for lines written to the log file. DEFAULT "*[time] [url] - [site]: [title] ([total])*". The valid field *[variables]* are: [stream]: The url of the first/best stream found. [site]: The site (submodule) name matching the webpage url. [url]: The url searched for streams. [time]: Perl timestamp when the line was logged. [title], [artist], [album], [description], [year], [genre], [total], [albumartist]: The corresponding field data returned (or "*-na-*", if no value). $station->get(['playlist']) Returns an array of strings representing all stream URLs found. If *"playlist"* is specified, then an extended m3u playlist is returned instead of stream url(s). NOTE: For podcast sites, if an author / channel page url is given, rather than an individual podcast episode's url, get() returns the first (latest?) podcast episode found, and get("playlist") returns an extended m3u playlist containing the urls, titles, etc. for all the podcast episodes found on that page url from latest to oldest. $station->getURL([*options*]) Similar to get() except it only returns a single stream representing the first valid stream found. Current options are: *"random"*, *"nopls"*, and *"noplaylists"*. By default, the first ("best"?) stream is returned. If *"random"* is specified, then a random one is selected from the list of streams found. If *"nopls"* is specified, and the stream to be returned is a ".pls" playlist, it is first fetched and the first entry (or a random entry if *"random"* is specified) is returned. This is needed by Fauxdacious Mediaplayer. If *"noplaylists"* is specified, and the stream to be returned is a "playlist" (either .pls or .m3u? extension), it is first fetched and the first entry (or a random entry if *"random"* is specified) in the playlist is returned. $station->count() Returns the number of streams found for the station. $station->getStationID(['fccid']) Returns the station's site ID (default), or station's FCC call-letters ("fccid") for applicable sites and stations. $station->getTitle(['desc']) Returns the station's title, (or long description, if "desc" specified). NOTE: Some sights do not support a separate long description field, so if none found, the standard title field will always be returned. $station->getIconURL(['artist']) Returns the URL for the station's "cover art" icon image, if any. Some video and podcast sites will also provide a separate artist/channel icon. If 'artist' is specified, this icon url is returned instead, if any. $station->getIconData(['artist']) Returns a two-element array consisting of the extension (ie. "png", "gif", "jpeg", etc.) and the actual icon image (binary data), if any. This makes it easy to download the image to local storage for use by your preferred media player. Some video and podcast sites will also provide a separate artist/channel icon. If 'artist' is specified, this icon's data is returned instead, if any. $station->getImageURL(['artist']) Returns the URL for the station's "cover art" banner image, if any. NOTE: If no "banner image" (usually a larger image) is found, the "icon image" URL will be returned. Some video and podcast sites will also provide a separate artist/channel image (usually larger). If 'artist' is specified, this icon url is returned instead, if any. $station->getImageData(['artist']) Returns a two-element array consisting of the extension (ie. "png", "gif", "jpeg", etc.) and the actual station's banner image (binary data). This makes it easy to download the image to local storage for use by your preferred media player. NOTE: If no "banner image" (usually a larger image) is found, the "icon image" data, if any, will be returned. Some video and podcast sites will also provide a separate artist/channel image (usually larger). If 'artist' is specified, this icon's data is returned instead, if any. $station->getType() Returns the station / podcast / video's type (*submodule-name*). (one of: "Anystream", "Apple", "BitChute", "Blogger", "Youtube", etc. - depending on the sight that matched the URL). Some video and podcast sites will also provide a separate artist/channel image (usually larger). If 'artist' is specified, this icon url is returned instead, if any. CONFIGURATION FILES The default root location directory for StreamFinder configuration files is "~/.config/StreamFinder". To use an alternate location directory, specify it in the "*STREAMFINDER*" environment variable, ie.: $ENV{STREAMFINDER} = "/etc/StreamFinder". ~/.config/StreamFinder/config Optional text file for specifying various configuration options. Each option is specified on a separate line in the formats below: NOTE: Do not follow the lines with a semicolon, comma, or any other separator. Non-numeric *values* should be surrounded with quotes, either single or double. Blank lines and lines beginning with a "#" sign as their first non-blank character are ignored as comments. 'option' => 'value' [, ...] 'option' => ['value1', 'value2', ...] [, ...] 'option' => {'key1' => 'value1', 'key2' => 'value2', ...} [, ...] and the options are loaded into a hash used by all sites (submodules) that support them. Valid options include *-debug* => [0|1|2] and most of the LWP::UserAgent options. ~/.config/StreamFinder/*submodule*/config Optional text file for specifying various configuration options for a specific site (submodule, ie. "Youtube" for StreamFinder::Youtube). Each option is specified on a separate line in the formats below: 'option' => 'value' [, ...] 'option' => ['value1', 'value2', ...] [, ...] 'option' => {'key1' => 'value1', 'key2' => 'value2', ...} [, ...] and the options are loaded into a hash used only by the specific (submodule) specified. Valid options include *-debug* => [0|1|2] and most of the LWP::UserAgent options. NOTE: Options specified here override any specified in *~/.config/StreamFinder/config*. NOTE: Options specified in the options parameter list of the *new()* function will override those corresponding options specified in these files. DEPENDENCIES URI::Escape, HTML::Entities, LWP::UserAgent RECCOMENDS youtube-dl, or other compatable program such as yt-dlp, etc. (for Youtube, Bitchute, Blogger, Brighteon, Odysee, Vimeo) NOTE: Required for Youtube, Odysee, and SoundCloud to work. wget BUGS Please report any bugs or feature requests to "bug-streamFinder at rt.cpan.org", or through the web interface at . I will be notified, and then you'llautomatically be notified of progress on your bug as I make changes. SUPPORT You can find documentation for this module with the perldoc command. perldoc StreamFinder You can also look for information at: SEE ALSO Fauxdacious media player - () * RT: CPAN's request tracker (report bugs here) * CPAN Ratings * Search CPAN LICENSE AND COPYRIGHT Copyright 2017-2023 Jim Turner. This program is free software; you can redistribute it and/or modify it under the terms of the the Artistic License (2.0). You may obtain a copy of the full license at: Any use, modification, and distribution of the Standard or Modified Versions is governed by this Artistic License. By using, modifying or distributing the Package, you accept this license. Do not use, modify, or distribute the Package, if you do not accept this license. If your Modified Version has been derived from a Modified Version made by someone other than you, you are nevertheless required to ensure that your Modified Version complies with the requirements of this license. This license does not grant you the right to use any trademark, service mark, tradename, or logo of the Copyright Holder. This license includes the non-exclusive, worldwide, free-of-charge patent license to make, have made, use, offer to sell, sell, import and otherwise transfer the Package with respect to any patent claims licensable by the Copyright Holder that are necessarily infringed by the Package. If you institute patent litigation (including a cross-claim or counterclaim) against any party alleging that the Package constitutes direct or contributory patent infringement, then this Artistic License to you shall terminate on the date that such litigation is filed. Disclaimer of Warranty: THE PACKAGE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS "AS IS' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT ARE DISCLAIMED TO THE EXTENT PERMITTED BY YOUR LOCAL LAW. UNLESS REQUIRED BY LAW, NO COPYRIGHT HOLDER OR CONTRIBUTOR WILL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING IN ANY WAY OUT OF THE USE OF THE PACKAGE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.