Comparing Dataframes: Finding Rows Present in One but Not in the Other

Comparing dataframes to identify differences is crucial for data quality assurance and merging operations. In this case, we have two dataframes (df1 and df2) with a specific structure and need to determine the rows present in df2 but not in df1.

Initially, attempts to compare dataframes using df1 != df2 resulted in an error. This approach only works for dataframes with identical rows and columns. To find symmetric differences, we need a different approach.

One method involves concatenating the dataframes:

df = pd.concat([df1, df2])
df = df.reset_index(drop=True)

Then, grouping the concatenated dataframe by all columns:

df_gpby = df.groupby(list(df.columns))

Next, we identify the unique records by obtaining the index values where only one row exists:

idx = [x[0] for x in df_gpby.groups.values() if len(x) == 1]

Using these indices, we can filter the dataframe to obtain the desired result:

df.reindex(idx)

This approach provides the rows present in df2 but absent in df1 based on the comparison of the Date index and the Fruit column.