6e. Illustration - Johnny, the YouTuber

Introduction

Through this section's illustration, we'll show the convenience of the following features:

1. Boolean indexing for working with subsets of the data

2. organizing code around functions

3. pipes to enhance code readability

4. use of views to modify subsets of the data

Some General Information

We begin by presenting some information about the per-view payments made by YouTube. We first confirm that Johnny's payments ranged from $2 to $6. Moreover, using the countmaps function from the StatsBase package, we conclude that Johnny has eight videos reaching the maximum payment of $6.

range_payrates  = unique(payrates) |> sort

5-element Vector{Int64}:
 2
 3
 4
 5
 6

using StatsBase
occurrences_payrates = countmap(payrates) |> sort

OrderedDict{Int64, Int64} with 5 entries:
  2 => 4
  3 => 5
  4 => 8
  5 => 5
  6 => 8

We can also provide some information about Johnny's most profitable videos. By applying the sort function and isolating the top 3 videos, we can obtain information on the highest earnings videos. Moreover, we can apply the sortperm function to identify the indices of these videos, allowing us to extract the payment per view and total Visits associated with each of them.

top_earnings    = sort(earnings, rev=true)[1:3]

3-element Vector{Float64}:
 2708.57
 1083.07
  723.493

indices         = sortperm(earnings, rev=true)[1:3]

sorted_payrates = payrates[indices]

3-element Vector{Int64}:
 6
 5
 5

indices         = sortperm(earnings, rev=true)[1:3]

sorted_visits   = visits[indices]

3-element Vector{Float64}:
 451.428
 216.615
 144.699

Functions to Represent Tasks

The approach employed so far allows for a rapid exploration of Johnny's viral videos. However, it falls short in providing a systematic analysis that could be extended to other subsets of data. To overcome this limitation, we can automate the process by defining a function.

Recall that a well-designed function should embody a specific task, implying that it must be independent of its specific application. In our case, the goal is to subset data and extract specific statistics, including the number of videos, visits, and revenue generated.

The function below implements this task taking three arguments: the raw data (visits and payrates) and a condition defining the subset (condition). By keeping the condition generic, we can seamlessly apply the same analysis to various subsets of data. The example also showcases the convenience of pipes to compute intermediate temporary steps, using it to retrieve the income earned from a subset of videos.

#
function stats_subset(visits, payrates, condition)
    nrvideos = sum(condition)
    audience = sum(visits[condition])
    
    earnings = visits .* payrates
    revenue  = sum(earnings[condition])
    
    return (; nrvideos, audience, revenue)
end

using Pipe
function stats_subset(visits, payrates, condition)
    nrvideos = sum(condition)
    audience = sum(visits[condition])
    
    
    revenue  = @pipe (visits .* payrates) |> x -> sum(x[condition])
    
    return (; nrvideos, audience, revenue)
end

using Pipe
function stats_subset(visits, payrates, condition)
    nrvideos = sum(condition)
    audience = sum(visits[condition])
    
    
    revenue  = @pipe (visits .* payrates) |> sum(_[condition])
    
    return (; nrvideos, audience, revenue)
end

Below, we illustrate how the function enables effortlessly characterizing various subsets of data.

viral_threshold  = 100
is_viral         = (visits .≥ viral_threshold)
viral            = stats_subset(visits, payrates, is_viral)

(nrvideos = 6, audience = 1243.63, revenue = 6133.19)

viral_threshold  = 100
is_notviral      = .!(is_viral)      # '!' is negating a boolean value and we broadcast it
notviral         = stats_subset(visits, payrates, is_notviral)

(nrvideos = 24, audience = 1169.13, revenue = 4971.02)

days_to_consider = (1, 10, 25)      # days when the videos were posted
is_day           = in.(eachindex(visits), Ref(days_to_consider))
specific_days    = stats_subset(visits, payrates, is_day)

(nrvideos = 3, audience = 118.547, revenue = 414.113)

Broadcasting Over a List of Functions (OPTIONAL)

At the beginning of the analysis, we could've derived descriptive statistics to gain insights about Johnny's videos. This can be accomplished by using the describe function from the StatsBase package. Despite this, the presentation aims to highlight that functions are first-class objects in Julia. This property entails that a function behaves just like any other variable, allowing the user to define a list of functions and then apply them element-wise to a variable.

list_functions = [sum, median, mean, maximum, minimum]

stats_visits   = [fun(visits) for fun in list_functions]

5-element Vector{Float64}:
 2661.48
   52.7884
   88.716
  541.714
   27.7205

By broadcasting the operation, we can also compute stats for multiple variables concurrently. For instance, below we characterize visits and earnings simultaneously.

list_functions = [sum, median, mean, maximum, minimum]

stats_various  = [fun.([visits, payrates]) for fun in list_functions]

5-element Vector{Vector{Float64}}:
 [2661.48, 128.0]
 [52.7884, 4.0]
 [88.716, 4.26667]
 [541.714, 6.0]
 [27.7205, 2.0]

One major limitation of the current method is its inability to capture each statistic's name. To overcome this, we can employ a named tuple, which we'll call stats_visits. The approach enables us to access stats through their respective names, such as stats_visits.mean or stats_visits[:mean] for the average value.

The implementation is based on the type Symbol. This converts strings into identifiers, which are necessary to programmatically access a named tuple's keys.

vector_of_tuples = [(Symbol(fun), fun(visits)) for fun in list_functions]
stats_visits     = NamedTuple(vector_of_tuples)

(sum = 2661.48, median = 52.7884, mean = 88.716, maximum = 541.714, minimum = 27.7205)

88.716

52.7884